Deploy a Celery Worker

In the course of processing web requests, you may have to offload tasks to an asynchronous, background process (typically called a worker). Render makes this trivial to do through Background Workers, which work like any other service.

For this quick start, we’ll use Celery, a popular distributed task queue for Python with a Flask frontend to submit sample tasks. We will also deploy Flower to view tasks as they’re processed by Celery.

Celery can use Redis for its processing backend, so we’ll set up a private Redis service first.

1. Create a new private Redis server using the Redis deployment guide and make a note of the service address which will look like redis:10000.

2. Fork render-examples/celery on GitHub and create a new Background Worker on Render using your fork.

   Environment	Python
   Build Command	pip install -r requirements.txt
   Start Command	celery worker -A tasks -l info

   Add the following environment variable under Advanced:

   Key	Value
   CELERY_BROKER_URL	redis://<redis-url>, where redis-url is the URL from step 1.

3. Create a new Web Service using the same repo you created in Step 2.

   Environment	Python
   Build Command	pip install -r requirements.txt
   Start Command	gunicorn app:app

   Add the following environment variable under Advanced:

   Key	Value
   CELERY_BROKER_URL	redis://<redis-url>, where redis-url is the URL from step 1.

   This will spin up a new Flask web service which simply lets you add two numbers as an asynchronous task.

4. Set up a Flower instance to monitor Celery. Flower is a web based tool to monitor and administer Celery. You can use it to view tasks and results, which is exactly what we’ll do in this section. Let’s start by creating a new Web Service using the same repo.

   Environment	Python
   Build Command	pip install -r requirements.txt
   Start Command	celery flower -A tasks -l info

   Add the following environment variable under Advanced:

   Key	Value
   CELERY_BROKER_URL	redis://<redis-url>, where redis-url is the URL from step 1.

   Once the service is live, navigate to the service URL to view tasks you’ve sent from the Python frontend.

   Note: Flower’s dashboard is not meant for public consumption, so we recommend securing it by adding authentication as described in the Flower docs.

You can also use a Celery background worker from Django. See our Django quick start and the Django integration guide in Celery docs.