Deploy a Celery Worker


In the course of processing web requests, you may have to offload tasks to an asynchronous, background process (typically called a worker). Render makes this easy to do through Background Workers, which work like any other service.

For this quick start, we’ll use Celery, a popular distributed task queue for Python with a Flask frontend to submit sample tasks. We will also deploy Flower to view tasks as they’re processed by Celery.

Celery can use Redis for its processing backend, so we’ll set up a private Redis service first.

  1. Create a new private Redis server using the Redis deployment guide and make a note of the service address which will look like redis:10000.
  2. Fork render-examples/celery on GitHub and create a new Background Worker on Render using your fork.

    EnvironmentPython
    Build Commandpip install -r requirements.txt
    Start Commandcelery worker -A tasks -l info

    Add the following environment variable under Advanced:

    KeyValue
    CELERY_BROKER_URLredis://<redis-url>, where redis-url is the URL from step 1.
  3. Create a new Web Service using the same repo you created in Step 2.

    EnvironmentPython
    Build Commandpip install -r requirements.txt
    Start Commandgunicorn app:app

    Add the following environment variable under Advanced:

    KeyValue
    CELERY_BROKER_URLredis://<redis-url>, where redis-url is the URL from step 1.

    This will spin up a new Flask web service which simply lets you add two numbers as an asynchronous task.

  4. Set up a Flower instance to monitor Celery. Flower is a web based tool to monitor and administer Celery. You can use it to view tasks and results, which is exactly what we’ll do in this section. Let’s start by creating a new Web Service using the same repo.

    EnvironmentPython
    Build Commandpip install -r requirements.txt
    Start Commandcelery flower -A tasks -l info

    Add the following environment variable under Advanced:

    KeyValue
    CELERY_BROKER_URLredis://<redis-url>, where redis-url is the URL from step 1.

    Once the service is live, navigate to the service URL to view tasks you’ve sent from the Python frontend.

    Note: Flower’s dashboard is not meant for public consumption, so we recommend securing it by adding authentication as described in the Flower docs.

You can also use a Celery background worker from Django. See our Django quick start and the Django integration guide in Celery docs.