Data processor workflow

Why deploy data processor workflow on Render?

A data processor workflow is a pipeline that transforms, merges, or enriches data from multiple sources into a unified output. This demo solves the problem of combining fragmented customer data (CRM, billing, product, support) into enriched profiles using parallel processing for high throughput.

This template wires together a Next.js frontend and FastAPI backend with the environment variables and service connections already configured, so you skip the setup of cross-service communication and secrets management. The pre-configured Render Workflow integration means you get parallel shard-based processing working immediately—no need to manually set up the SDK, define workflow slugs, or debug API key scoping. One-click deploy gets you a working customer data merge demo in minutes instead of the hour-plus it would take to configure services, environment variables, and workflow triggers from scratch.

Architecture

What you can build

After deploying, you'll have a working demo that merges customer records from four data sources (CRM, billing, product, and support) into unified profiles with calculated health scores and churn risk indicators. The frontend lets you trigger the workflow and watch it process 400K sample records across 10 parallel shards, with a real-time event log showing progress. You can use this as a reference implementation for building your own parallel data processing pipelines on Render Workflows.

Key features

• Hash-based parallel sharding: Routes records to 10 parallel shards using deterministic hashing on customer_id, ensuring same customer data always processes together across all source files.
• Multi-source data merging: Combines four CSV sources (CRM, Billing, Product, Support) into enriched customer profiles with calculated health_score, churn_risk, and expansion_potential.
• Dual language implementations: Provides identical workflow implementations in both Python (FastAPI + render_sdk) and TypeScript (Fastify + @renderinc/sdk) with matching APIs.
• Local workflow development: Render CLI runs a local task server on port 8120, enabling full workflow testing with RENDER_USE_LOCAL_DEV flag before deployment.
• Blueprint deployment ready: Includes render.yaml for one-click deployment of frontend and API services, with manual workflow creation instructions for the processing layer.

Use cases

• Data team merges CRM, billing, and support records into unified customer profiles
• Platform engineer processes 400K records in parallel to calculate churn risk scores
• Analytics team enriches customer data from four sources to identify expansion opportunities
• Backend developer shards large datasets for parallel processing across multiple workers

What's included

Prerequisites

• Render API Key: Your Render API key used to authenticate and trigger workflows from the API services.

Next steps

1. Open the frontend URL and click Run Workflow — You should see the EventLog populate with shard processing messages and a ResultsSummary showing 10 parallel shards completed with timing stats
2. Test the Python API health endpoint at /docs — You should see the FastAPI Swagger documentation with the trigger endpoints listed and be able to execute a test request
3. Configure the WORKFLOW_SLUG environment variable on the API service to match your workflow name — When you trigger a workflow from the frontend, you should see it appear in the Render Dashboard under Workflows with status Running or Completed

Resources

• Render Workflows official site