How to trigger a long-running task from a web service on Render
Architecture of asynchronous handoffs
Web applications frequently need to ingest incoming requests instantly while deferring heavy computation to background processes. This separation prevents bottlenecks when handling computationally expensive workloads. On Render, you achieve this asynchronous handoff by decoupling your architecture into two components, typically organized within the same Render project. A web service acts as the public-facing ingestion point, and a workflow service handles the deferred execution.
Bridging these two components requires familiarity with REST API design and webhook consumption patterns. To review the structural differences between async tools before building your implementation, read cron jobs vs background workers vs workflows: picking the right async primitive.
This guide focuses on architecting an asynchronous handoff between a public web service and a backend workflow. You learn payload management and idempotency constraints so you can adapt them for your production environment.
Why you must not await long tasks
A common anti-pattern when building API endpoints is attempting to execute long-running operations directly inside the web request lifecycle. Awaiting a video processing job or PDF report generation inside your request handler leads to timeout errors and dropped client connections. Learn more about the rationale for separating these concerns in serverless functions vs workflows: where long-running tasks should live.
Every web request passes through multiple networking layers, including proxies and browsers that drop idle connections. A deploy or restart of your web service kills any in-flight work. Webhook senders time out client-side in seconds (for example, GitHub times out after 10 seconds) and retry. Many HTTP clients and libraries implement this timeout-based retry logic that fires if a response isn't received within a configured window. If this occurs, they treat the request as failed and might trigger a retry, even though the server-side process continues executing.
This disconnect creates cascading failures. External providers like Stripe or GitHub require rapid acknowledgment when they send webhooks. If your server holds the connection open while generating an invoice, the external provider assumes the delivery failed and resends the exact same event. Because your server is still processing the initial request, you now have duplicate processes running simultaneously.
Decoupling web request handlers from long-running execution prevents these timeout errors. Your web service must act as a traffic cop. Its only responsibility is routing instructions to the workflow service and immediately acknowledging the receipt of the payload.
Designing the thin handler and triggering tasks
To implement the traffic cop pattern, use the "thin handler" approach. The thin handler delegates all heavy processing to Render Workflows by immediately returning an HTTP 202 Accepted status alongside a unique task run identifier. The 202 Accepted status communicates that you received the request but the processing is not yet complete.
You can trigger a task from your thin handler by using the Render SDK for TypeScript to programmatically queue the job or by calling the Run task API endpoint directly. Both approaches interact with the Render Workflows API and require orchestration.
When interacting with the Render API, you must anticipate and handle platform constraints. The Render API's Run task endpoint limits each Render user to 100 requests per minute. If your web service attempts to trigger 150 workflow tasks simultaneously, the API responds with a 429 Too Many Requests status. Render does not queue the run on a 429 response, so a trigger that is not retried is a lost event. Handling this requires retry logic with exponential backoff and jitter. If your queued tasks spin up and connect to a database, you must manage connection pooling so that 200 concurrent task runs don't meet your Postgres connection limit.
Render Workflows operate as task runs with automatic per-task retries. The platform does not offer mechanisms for pausing execution or resuming partially completed execution states.
While the upcoming code block demonstrates an inline SDK call for educational clarity, a production architecture often uses a transactional outbox pattern. This pattern saves the incoming webhook to a database table and relies on a separate internal worker to poll the database and trigger the Render API.
You can trigger a task by calling the Run task API directly via REST. You issue a POST request to the API containing your task name and input arguments:
Managing payload size and idempotency
Distributed systems require payload management to guarantee data integrity across asynchronous boundaries. When passing instructions from your web service to a Render workflow, you are restricted by platform limitations. Render Workflows enforce a 4 MB limit for the combined size of all arguments passed to a single task run.
Passing large raw video files or nested JSON arrays directly as argument strings causes the task trigger to fail. To bypass this constraint, implement the payload-by-reference pattern.
Instead of pushing the entire data object into the workflow queue, your thin handler saves the large incoming webhook payload to a durable storage medium like a managed PostgreSQL database. Local filesystems attached to web services are ephemeral and inappropriate for this persistent data handoff. Once you save the payload, the handler extracts the unique database identifier and passes only this string to the Render workflow task.
This architectural pattern also solves idempotency challenges. Because external platforms often retry webhook deliveries during network partitions, your workflow must handle duplicate events.
To achieve idempotent retry behavior, extract a unique webhook identifier provided by the external sender. You then enforce atomic uniqueness constraints on your database table using this identifier. If a duplicate webhook arrives, the database rejects the insertion, preventing the workflow from processing the same event twice.
This conceptual snippet illustrates saving large payloads before triggering the workflow, avoiding the 4 MB argument limit:
Checking status and callbacks
Once your thin handler delegates the payload reference to the background task, the original client application often needs a mechanism to track the execution progress. Because the initial HTTP connection closes with a 202 Accepted response, clients must rely on secondary communication channels.
The first approach involves client-side polling. The thin handler responds with the unique task run ID generated by the Render platform. The client application then periodically sends HTTP requests to a dedicated status endpoint on your web service.
If you check the status via the Render API, render.workflows.getTaskRun(taskRunId) returns a status of pending, running, completed, failed, or canceled. The REST equivalent is the Retrieve task run endpoint:
A status endpoint that proxies getTaskRun on every client poll quickly spends your GET API budget (400 requests per minute per user). We suggest reading the status from the database row the task already updates instead.
The preferred approach is the asynchronous callback pattern. In this design, the initial client request includes a destination webhook URL. The web service passes this callback URL along with the payload reference to the workflow service. The workflow task performs its computational duties and, upon completion, issues an outgoing HTTP POST request back to the client's provided URL. This push-based model eliminates redundant polling requests and reduces the processing load on your primary web service.
A simplified version of the workflow task might look like this:
Common mistakes and troubleshooting
When architecting asynchronous background pipelines, you might encounter specific operational pitfalls. A misconception is assuming that Render Workflows possess built-in capabilities to pause execution or resume from arbitrary points of failure. They do not. Each task run is retried as a whole unit. If your task's code fails at the ninety percent mark, the retry re-executes that task's function from its first line. Your background application code must be idempotent to accommodate these platform-level retries.
Another error involves failing to implement backoff logic for 429 rate limit responses. The Render API's Run task endpoint limits each Render user to 100 requests per minute. When sudden traffic spikes occur, omitting exponential backoff mechanisms causes you to drop incoming events, because Render does not queue the run on a 429 response. Implementing randomized jitter alongside your retry loops prevents the thundering herd problem, where multiple delayed tasks attempt to reconnect.
You might mistakenly pass large raw files or JSON arrays directly into the task arguments. Bypassing the argument size limits requires using the payload-by-reference architecture. Storing large payloads in your database guarantees you never exceed the 4 MB constraints.
If you experience unexpected behavior during your implementation, review the official Troubleshooting Deployments documentation to familiarize yourself with general debugging patterns. Monitoring your service logs and validating your API authentication headers resolves most initial integration challenges.