At localhost, Gauthami Polasani, founding PMM at LlamaIndex, walked through a reference architecture for document processing: LlamaIndex handling document intelligence, Render handling execution. LlamaIndex started as an open source RAG framework, then spent the last two years going deep on one specific problem underneath that: document processing.
Most company knowledge isn't sitting in a database
Every team building agents wants them to reason over the company's own knowledge, and most of that knowledge lives in PDFs, scanned contracts, spreadsheets, and slide decks, not in clean rows in a database. Getting an agent to reason over any of it depends entirely on parsing and extracting it correctly first and that step is where most document pipelines break.
A standard PDF isn't a structured file the way JSON or HTML is. To code, it's closer to a digital printout: nothing marks where a paragraph starts, where a table ends, or what the actual reading order is supposed to be. A basic text extractor doesn't see any of that. It reads left to right, straight across the page, regardless of the visual layout underneath. A two-column layout or a sidebar gets flattened and rotated into the wrong order, stitching unrelated blocks of text together into a scrambled mess that ruins the context before a model ever sees it. And a real production pipeline isn't just PDFs: it's forms, scanned invoices, phone photos of a whiteboard, all hitting the same upload path, which makes hand-written parsing logic for every variation a losing bet at scale.
Frontier models handle a simple, text-only document fine. The moment tables or multi-column layouts enter the picture, accuracy drops, because these models are built to reason and not to reconstruct visual structure.
Throwing more tokens isn't going to make your accuracy better. It just makes your pipeline more expensive.
There's an operational gap underneath that, too. Most model APIs hand back flat text: no bounding boxes, no confidence scores, no provenance. If a model misreads a number out of a table or a chart, there's nothing in that response to catch it, and no way to go back and ask what data drove that answer.
Classification, parsing, and extraction are different jobs
LlamaParse splits document ingestion into three separate stages instead of one black box. Classification figures out what kind of document arrived (an invoice, a contract, something else) and returns a baseline confidence score, which matters because an invoice and a contract need different extraction logic downstream. Parsing turns that classified document into readable content: instead of scraping blindly, it first works out the visual layout, maps the columns, preserves tables, runs OCR if the source is a scan, and outputs a clean Markdown a downstream model can read. Extraction takes that Markdown and pulls the exact fields a schema asks for, a vendor name, a balance, whatever's needed, into structured JSON.
Collapsing those three stages into one step hides where a pipeline fails. A misclassified invoice drags the wrong extraction rules along with it. A table that gets flattened badly means a perfectly correct schema still lands on the wrong values. Extraction failures don’t necessarily mean the parser needs to run again, and rerunning it blindly can waste time without ever touching the real problem. Keeping the stages separate makes all of that inspectable: the document type and its confidence, the parsed Markdown, and the extracted JSON can each be checked against the source on their own.
Keep the upload path thin
The reference architecture runs as three services on Render. A thin Express web service has one job: accept the file, hand it straight to the workflow, and hold an SSE connection open so the UI can stream progress back live. It never touches document processing itself. Render Workflows owns that entire processing sequence instead, without the message brokers and standing infrastructure that reliable background jobs would normally require. Each workflow defines five tasks covering the steps a document goes through: storing the file, classifying it, parsing it, extracting the requested fields, and storing the results. LlamaParse handles the document understanding inside those steps; Render handles spinning up the execution around them.
Store the intermediate truth
Everything lands in Render Postgres across four tables: document metadata, classification output with its confidence score, parsed Markdown, and extracted JSON, all fully queryable and auditable on their own. That's the audit trail for a wrong field: did classification pick the wrong document type, did parsing lose the relationship between a heading and the table underneath it, or did extraction map a correct parse to the wrong field in the schema? A single final JSON blob can't answer any of that on its own; the intermediate stages sitting next to it can. Confidence scores help route an uncertain document toward review, but they don't certify that a classification or an extraction is correct.
The demo ran two documents through the same pipeline: a short, visually irregular receipt with three line items, a subtotal, and sales tax, and a 24-page commercial bank statement full of account details, balances, and transactions.
The receipt came back classified as a receipt at 85 percent confidence, with the reasoning behind that call visible alongside the extracted items, store name, subtotal, and tax. The bank statement came back classified as a financial document at 100 percent confidence, reasoning tied to the standard elements it found (account holder name and the rest), all 24 pages parsed, and five fields extracted, balance, credits, and transaction ranges among them.
Either way, the classification and the parsed Markdown stay right next to the final JSON. If a number in that JSON is wrong, there's a clear place to go look for why.