Asynchronous tasks with Celery
Vatsal Parekh (~vatsal) |
Abstract : This talk is about to demonstrate use and features of async task queue with Celery and Rabbitmq. Also deploying it on multi-node architecture.
Description : Responding to web calls that include things not necessary to be done at the time of the call, or doing tasks that are relatively longer time consuming and are asynchronous in nature, can be better deployed with an async task queue mechanism.
Celery is a very flexible async task queue, that can also be deployed distributedly on multi-node architecture. It can be used within any small Python project, or even for a bigger environment like Django. The official doc goes like
Celery: Distributed Task Queue Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on the real-time operation but supports scheduling as well. The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or event. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
Intro to Async Tasks
Up and running with Celery
"Hello World" on celery
Advanced Celery Configurations
Scheduled Async tasks with Celerybeat
Running Celery on multi-node architecture
Basic knowledge of Python
Working installation of Python, rabbitmq and celery
I'm a Python Developer, like to work on Django for my web related stuff, trying to develop things that work at large scale fascinate me.
Currently working on a Twitter Automation tool called Twito. Previously worked with Oizom Instruments to develop their data-platform.