Scheduling Tasks in Django Django.How

Purpose

There are long running tasks that you want to hand off to a background process without it blocking the page from loading

While we wait for it to complete.

Like sending emails and generating huge files and graphics.

Way 1: django-background-tasks

1. install

pip install django-background-tasks

2. Add to INSTALLED_APPS:\

INSTALLED_APPS = (
# …
‘background_task’,
# …
)

3. Migrate your database


python manage.py migrate

4. Creating tasks

from background_task import background
from django.contrib.auth.models import User

@background(schedule=60)
def notify_user(user_id):
    # user_id and any other parameters must all be serializable as JSON
    user = User.objects.get(pk=user_id)
    user.email_user('Here is a notification', 'You have been notified')

5. registering (calling) tasks


notify_user(user.id) # default time will be taking from the decorator
notify_user(user.id, schedule=90) # 90 seconds from now
notify_user(user.id, schedule=timedelta(minutes=20)) # 20 minutes from now
notify_user(user.id, schedule=timezone.now()) # at a specific time
notify_user.now(user.id)   # launch a notify_user function and wait for it
notify_user = notify_user.now   # revert task function back to normal function. Useful for testing.
notify_user(user.id, verbose_name="Notify user", creator=user) # specify a verbose name and a creator when scheduling a task

6. Running tasks


python manage.py process_tasks

Resources:

Way 2: Django Management command and cron tab

https://django.cowhite.com/blog/scheduling-taks-in-django/
https://devcenter.heroku.com/articles/scheduling-custom-django-management-commands

Way 3: task as a regular Python thread


def start_new_thread(function):
    def decorator(*args, **kwargs):
        t = Thread(target = function, args=args, kwargs=kwargs)
        t.daemon = True
        t.start()
    return decorator

@start_new_thread
def foo():

do stuff

https://stackoverflow.com/questions/18420699/multithreading-for-python-django

Way 4: Celery Most Advanced

a well maintained and popular Python package that can be used in Django without any Django-specific extensions.

Notes:

Celery is excellent, I use it my self , but it has a steep learning for setting up tasks
also you will be need to use message broker to send tasks to celery and additional utility to run celery in daemon mode.
AWS Elastic Beanstalk, answer should be yes
Celery is a bit more structured and wants you to store these tasks in a tasks.py file in the application
Celery and Django-Q both have extensive documentation
Require broker like (Redis)
use decorators with the task so all your async imports and logic are within the tasks.py file.
Celery has it’s own logging functionality which is imported and appears to work just fine.

Resources:

Way 5: Django-Q

a task queue, scheduler and worker application designed specifically for Django.

Require broker like (Redis)
Django-Q and Django-RQ both just use standard Python logging, which I preferred for my use case.
Celery and Django-Q both have extensive documentation

Resources:

https://stuartm.com/2020/05/basic-django-q-example/

Way 6: Django-RQ

a Django extension for the popular RQ (Redis Queue) python library

Require broker like (Redis)
use decorators with the task so all your async imports and logic are within the tasks.py file.
Django-Q and Django-RQ both just use standard Python logging which I preferred for my use case.
No extensive documentation

Resources:

WAY of the future

All of the above may become irrelevant in the future when Django implements full view-based support for async code.

How are Django channels different than celery?

https://stackoverflow.com/questions/38620969/how-are-django-channels-different-than-celery

Channels:

Channels in Django is for WebSocket, long-poll HTTP.
In Django are meant for asynchronous handling of requests.
The standard model Django uses is Request-Response but that has significant limitations. We cannot do anything outside the restrictions of that model.Channels came about to allow Web Socket support and building complex applications around Web Sockets, so that we can send multiple messages, manage sessions, etc.
Django channels does support background processing. I think the question is more on, how django channels background processing is different from celery workers.
Django channels gives to Django the ability to handle more than just plain HTTP requests, including Websockets and HTTP2. Think of this as 2 way duplex communication that happens asynchronously No browser refreshing. Multiple clients can send and receive data via websocket and django channels orchestrates this intercommunication example a group chat with simultaneously clients accessing at the same time. You can achieve background processing of long running code simliar to that of a celery to a certain extent, but the application of channels is different to that of celery.
Passes messages between client and server (Full duplex connection)
Handle HTTP and Web-sockets requests
Asynchronous

(Example cases)

Real time chat application
Update social feeds
Multiplayer game
Sending Notifications

Celery:

Celery is for background task, queue.
is a completely different thing, it is an asynchronous task queue/job queue based on distributed message passing. It is primarily for queuing tasks and scheduling them to run at specific intervals.
Celery is an asynchronous task queue/job queue based on distributed message passing. As well as scheduling. In layman’s terms, I want to fire and run a task in the background or I want to have a periodical task that fires and runs in the back on a set interval. You can also fire task in a synchronous way as well fire and wait until complete and continue. So the key difference is in the use case they serve and objectives of the frameworks.
Perform long running background tasks
Perform periodic tasks
Asynchronous

(Example cases)

Processing Videos/images
Sending bulk emails

Both can

Channels & Celery can both do asynchronous pooled tasks in common.
Channels and Celery both use a backend for messages and worker daemon(s). So the same kind of thing could be implemented with both.

Resources: