Self-Orchestrating Agents - Python

Why deploy Self-Orchestrating Agents on Render?

SF Pulse is a reference application that demonstrates self-orchestrating agents using Render Workflows with the Render Python SDK. It uses a cron-triggered workflow to coordinate multiple async tasks (scraping, LLM extraction, database updates) for tracking San Francisco restaurant openings and events. The pattern solves the problem of orchestrating multi-step background jobs with observability and retry logic without managing separate job queue infrastructure.

This template wires together a FastAPI web service, daily cron job, PostgreSQL database, and Redis for real-time SSE fanout—all pre-connected with the right environment variables and health checks configured. You skip the manual work of setting up database connections, configuring cron schedules, and wiring Redis pub/sub across services. Render's managed Postgres and Redis mean no separate infrastructure to provision, and the cron service triggers workflows via the Render Python SDK without you building your own scheduler.

Architecture

What you can build

You'll have a web app that aggregates new San Francisco restaurant openings and local events, with a daily scheduled job that scrapes sources and optionally uses an LLM to extract structured data. The setup includes a PostgreSQL database, Redis for real-time updates via SSE, and a Render Workflow that orchestrates the daily data refresh—though the workflow service requires manual creation in the Dashboard after the initial deploy.

Key features

• Render Workflows SDK: Uses the Render Python SDK to orchestrate multi-step daily refresh workflows triggered by a cron service.
• Redis SSE fanout: Implements cross-instance Server-Sent Events broadcasting using Redis pub/sub for real-time updates.
• Provider-agnostic LLM extraction: Supports both OpenAI and Anthropic APIs for content extraction, with automatic provider detection from API key prefix.
• Asyncpg raw SQL: Uses asyncpg with plain SQL migrations instead of an ORM for direct PostgreSQL access.
• Testcontainers integration: Pytest suite spins up real PostgreSQL containers per session for integration testing.

Use cases

• Data team schedules daily scraping jobs to aggregate local event listings
• Startup founder automates multi-source content extraction with LLM processing
• Developer builds real-time dashboard refreshing from multiple external APIs
• DevOps engineer orchestrates sequential ETL tasks across web scrapers

What's included

Prerequisites

• LLM API Key: Your OpenAI or Anthropic API key used for AI-powered extraction of restaurant and event data from scraped content.
• Render API Key: API key for the Render platform, used by the cron service to trigger the daily refresh workflow.

Next steps

1. Open the web service URL and navigate to /diagram/ — You should see an interactive workflow diagram showing the daily-refresh pipeline stages
2. Trigger the sf-pulse-python-daily cron job manually from the Dashboard — The sf-pulse-python-workflow logs should show tasks executing for restaurant and event scraping within 30 seconds
3. Refresh the home page after the workflow completes — You should see San Francisco restaurant openings and local events populated on the page

Resources

• Render Workflows official site