Deploy a Celery Worker
In the course of processing web requests, you may have to offload tasks to an asynchronous, background process (typically called a worker). Render makes this easy to do through Background Workers, which work like any other service.
For this quick start, we’ll use Celery, a popular distributed task queue for Python with a Flask frontend to submit sample tasks. We will also deploy Flower to view tasks as they’re processed by Celery.
Celery can use Redis for its processing backend, so we’ll set up a private Redis service first.
- Create a new private Redis server using the Redis deployment guide and make a note of the service address which will look like
redis:10000
. -
Fork render-examples/celery on GitHub and create a new Background Worker on Render using your fork.
Environment Python
Build Command pip install -r requirements.txt
Start Command celery worker -A tasks -l info
Add the following environment variable under Advanced:
Key Value CELERY_BROKER_URL
redis://<redis-url>
, whereredis-url
is the URL from step 1. -
Create a new Web Service using the same repo you created in Step 2.
Environment Python
Build Command pip install -r requirements.txt
Start Command gunicorn app:app
Add the following environment variable under Advanced:
Key Value CELERY_BROKER_URL
redis://<redis-url>
, whereredis-url
is the URL from step 1.This will spin up a new Flask web service which simply lets you add two numbers as an asynchronous task.
-
Set up a Flower instance to monitor Celery. Flower is a web based tool to monitor and administer Celery. You can use it to view tasks and results, which is exactly what we’ll do in this section. Let’s start by creating a new Web Service using the same repo.
Environment Python
Build Command pip install -r requirements.txt
Start Command celery flower -A tasks -l info
Add the following environment variable under Advanced:
Key Value CELERY_BROKER_URL
redis://<redis-url>
, whereredis-url
is the URL from step 1.Once the service is live, navigate to the service URL to view tasks you’ve sent from the Python frontend.
Note: Flower’s dashboard is not meant for public consumption, so we recommend securing it by adding authentication as described in the Flower docs.
You can also use a Celery background worker from Django. See our Django quick start and the Django integration guide in Celery docs.
See Specifying a Python Version if you need to customize the version of Python used for your app.