July 10, 2026

Agents Don't Fail in the Demo. They Fail in Production. Architecture Is What Separates the Two.

Every week I see another agent demo. Someone wires a frontier model to a handful of tools, records a two-minute video of it booking a meeting or drafting a memo, and the comments fill up with "this changes everything."

Here's what those demos never show: the same agent on run four hundred, at 9 AM on a Tuesday, when the CRM API times out mid-task, the model gets throttled, and the output that looks perfectly plausible contains a portfolio figure that doesn't exist.

The demo was easy. The demo has always been easy. What separates a demo from a production system — the kind a wealth manager or family office can put in front of clients and regulators — is not the model. It's the architecture around the model. And most of the AI industry still isn't talking honestly about that gap.

MIT research found that roughly 95% of enterprise AI pilots produce zero measurable return, and as I argued in the piece on forward deployed engineers, those are overwhelmingly deployment failures, not model failures. Agentic workflows raise the stakes further, because agents don't just answer questions — they take actions, chain decisions, and touch real systems. When they fail, they fail in motion.

The math that kills naive agents

Start with the uncomfortable arithmetic. An agentic workflow isn't one model call — it's a chain of them: plan, retrieve, call a tool, interpret the result, decide, act, summarize. Each step has some probability of going wrong: a hallucinated field, a misread API response, a wrong branch.

Suppose each step is 99% reliable. That sounds excellent. Over a 30-step workflow, it compounds to roughly a 74% chance of a clean end-to-end run. At 95% per step — closer to reality for unaided model calls against messy enterprise data — a 20-step workflow completes cleanly about a third of the time.

No prompt tweak fixes compounding. You fix it the way engineers have always fixed unreliable components: you design a system that expects failure at every step, catches it, and recovers. That's an architecture problem. Which is good news, because architecture problems have known solutions.

Two failure modes matter most in an enterprise setting, and they need different machinery:

Durability failures — the workflow dies. An API times out, a provider throttles you, a process crashes at step 14 of 20, and the work is either lost or, worse, half-applied.

Accuracy failures — the workflow finishes and is wrong. The agent invents a fee schedule, cites a policy that doesn't exist, or confidently summarizes a document it never actually retrieved. In finance, this is the failure mode that ends up in front of a regulator.

The public record already shows what accuracy failures cost. A Canadian tribunal held Air Canada liable when its chatbot invented a bereavement-fare policy — the company's argument that the chatbot was "a separate legal entity responsible for its own actions" was rejected outright. Deloitte partially refunded the Australian government after a AU$440,000 report was found to contain AI-fabricated citations and a fabricated quote from a court judgment. Those weren't model problems. They were systems built without verification layers.

The control plane: durable execution comes first

The single most important architectural decision in an agentic system is this: the model does not own the control flow. The orchestration layer does.

A naive agent is a loop: model decides, model acts, model decides again, until it declares itself done. That design is fine for a demo and unacceptable for production, because there is no ground truth about where the workflow is, what it has already done, or how to resume it after a failure.

A production agentic workflow runs on a deterministic control plane — a workflow engine that owns sequencing, state, and recovery, and delegates bounded decisions to models. Concretely, that means:

Checkpointing and resumability. Every step's inputs and outputs are persisted. When step 14 fails, the system resumes at step 14 — it doesn't rerun the previous thirteen, and it doesn't lose them.
Idempotent actions with retries. Every side effect — a CRM update, a document filed, an email queued — is designed so that retrying it is safe. Timeouts and rate limits get exponential backoff, not a crashed pipeline.
Bounded autonomy. The model chooses within steps ("which of these accounts matches?"); the engine decides between steps. Loops have iteration caps. Every branch has a defined failure path.
Circuit breakers and fallbacks. When a provider is throttled or down, the router degrades gracefully to an alternate model instead of stopping the business. This is the same resilience argument behind hybrid local/cloud architecture: your briefings still go out when someone else's capacity planning fails.

None of this is exotic. It's the discipline distributed-systems engineers have applied to payments and trading infrastructure for decades, applied to a new kind of unreliable component. That's the point: agentic AI doesn't exempt you from engineering. It demands more of it.

The accuracy stack: how you actually mitigate hallucinations

Hallucination isn't a bug you patch. It's an intrinsic property of the component: language models produce plausible text, and plausibility is not truth. So you don't "fix" hallucination — you build a stack around the model that makes unsupported output unlikely to be generated, likely to be caught, and impossible to reach a client unreviewed. In practice, that stack has layers, and every layer earns its place.

Ground everything in retrieved truth. The model should answer from documents and data your system retrieved and handed to it — never from its own recollection of the world. Every factual claim in the output carries a citation back to a source the reviewer can check. If retrieval comes back empty, the correct output is "we don't have this," not a fluent guess. We design for abstention: an agent that says "I can't verify this" is behaving correctly, and the system rewards it.

Validate structure before trusting content. Agent outputs that feed downstream steps are never free text. They're structured outputs validated against schemas — typed fields, enumerated values, required citations. A number that should be a portfolio weight gets range-checked. An account identifier gets verified against the book of record, not pattern-matched. Malformed output fails loudly at the boundary instead of propagating silently.

Check claims against the source of truth. For high-stakes fields — figures, dates, names, policy language — the system programmatically verifies the generated value against the structured system it supposedly came from. This is cheap, deterministic code, and it catches the exact class of error that makes headlines.

Manage context like the scarce resource it is. Accuracy degrades when context is bloated, stale, or contradictory. Long-running agents accumulate conversational debris; models lose the thread in oversized contexts. Production systems curate aggressively: retrieval brings in only what the current step needs, summaries compact history between steps, and durable knowledge lives in a memory layer — compiled truth plus an append-only evidence timeline — rather than being re-derived from scratch every run. Context engineering, not prompt cleverness, is where most real-world accuracy gains come from.

Treat prompts as versioned, tested code. Prompts live in source control. Every change runs against a regression suite before it ships, because a wording tweak that improves one behavior routinely breaks another. The leaked system prompts from the major labs show how much deliberate structure the best teams put into this layer — none of it is improvised, and none of it should be at your firm either.

Budget tokens deliberately. Token management sounds like a cost concern, and it is — uncontrolled agents burn spectacular amounts of money. But it's also an accuracy and durability concern: per-step and per-workflow token budgets bound worst-case behavior, caching keeps repeated context cheap and consistent, and compaction keeps long workflows inside the window where models perform well. A system with no token discipline eventually fails in all three dimensions at once.

Route each task to the right model. Not every step needs a frontier model, and some steps must never leave your infrastructure. A routing layer sends deterministic, high-volume steps — classification, extraction, restricted-term screening — to small, fast, right-sized models, often local; reserves frontier models for genuine multi-step reasoning; and routes anything touching PII or client financial data inside your compliance boundary by default. Routing gives you accuracy where it matters, cost control everywhere else, and a fallback path when a provider has a bad day.

Independent verification: evals and evaluator agents

Everything above reduces the error rate. What catches the errors that remain is independence — and this is where finance firms should feel at home, because the principle is one your operations team already lives by: maker-checker. The one who does the work is never the one who signs off on it.

In an agentic architecture, that principle shows up twice.

Independent evals, run continuously. Before anything ships, it's measured against golden datasets — curated sets of real inputs with known-correct outputs, including the ugly edge cases: the ambiguous client request, the document with contradictory figures, the query about a security you don't cover. Every prompt change, model upgrade, and retrieval tweak runs the full suite, in CI, with gates — if faithfulness or citation accuracy drops, the change doesn't deploy. And because model providers silently update behavior, the same evals run on schedule in production to catch drift you didn't cause. If you can't measure whether yesterday's change made the system better or worse, you're not engineering — you're guessing with confidence.

Independent evaluator agents, in the loop. At runtime, a separate review agent — different prompt, ideally a different model, with zero stake in the generator's answer — audits output before it advances: Is every claim supported by the cited source? Does the math check? Does anything violate the firm's policy library? Fail, and the work goes back for revision with specific findings, or escalates to a human. The critical property is separation of duties: an agent reviewing its own work inherits its own blind spots. An independent evaluator with a narrow mandate — find what's wrong — reliably catches what the generator can't see. It's the compliance review desk, rebuilt in software, and running on every single output instead of a sample.

Humans at the gates that matter. Human-in-the-loop fails when it's applied indiscriminately — approve everything and people rubber-stamp; approve nothing and you've built liability. The answer is risk-tiered gates: internal, low-stakes, reversible actions run autonomously with logging; client-facing content and irreversible actions stop at a human approval gate, where the reviewer sees the output plus the evidence — citations, evaluator findings, confidence flags — so approval is a judgment, not a formality. Every decision is logged, and disagreements feed back into the eval suite. The goal was never to remove humans. It's to spend scarce human attention exactly where it changes the outcome.

Compliance is an architectural property, not a policy document

A compliance manual the agent can't read is not a control. In a production agentic system, compliance is enforced by the same machinery that runs the workflow:

A policy engine in the execution path. Restricted terms, required disclosures, and suitability rules are encoded as checks that run on every output — deterministically where possible, via evaluator agents where judgment is needed. Content that fails doesn't proceed. This isn't advisory; it's a gate.
Least-privilege tools. Agents get scoped credentials to the specific systems and operations they need — read access to research is not write access to the CRM. Risky operations run sandboxed. An agent that cannot perform an action cannot be talked into it.
An immutable audit trail. Every retrieval, model call, tool invocation, evaluator verdict, and human approval is logged with inputs, outputs, and versions of the prompt and model that produced them. When an auditor asks "why did the system say this?" you replay the exact decision chain — which model, which context, which sources, who approved. That question is coming to every regulated firm using AI. Architecture is what determines whether you can answer it.
Observability and drift monitoring. Traces, latency and cost dashboards, and quality metrics watched over time — because the failure you should fear most isn't the loud crash, it's the quiet degradation nobody notices until quarter-end.

What this looks like assembled

Take a workflow we build variants of for wealth managers: personalized, compliance-reviewed client briefings. In production form: the orchestration engine kicks off on schedule and checkpoints each stage. Retrieval assembles curated context per client — portfolio data from structured systems, market commentary, the firm's house views — through the memory layer. A local model classifies and screens; the router sends only the final personalization step to a frontier model, inside a token budget, with a fallback route if the provider is throttled. The draft comes back as structured output, citations required. Deterministic checks verify every figure against the book of record. An independent evaluator agent audits claims against sources and the policy library. Anything flagged loops back; everything else lands in an advisor's approval queue with the evidence attached. On approval it's delivered — and every step of that chain is in the audit log, replayable.

That system is durable: a provider outage delays nothing but the hardest 20% of the compute, and a crash resumes where it stopped. It's accurate: four independent layers stand between a hallucination and a client. And it's compliant by construction, not by promise.

Count the disciplines in that paragraph: workflow orchestration, retrieval engineering, schema design, model evaluation, security, observability, compliance encoding. That's the honest answer to why so many AI pilots die. The model was never the product. The system is the product — and systems require real engineering: senior people who have built failure-tolerant infrastructure before and understand the regulatory reality they're building inside. That's precisely the forward deployed work we do at West Stack, and it's why we keep saying the winners of enterprise AI won't be the firms with the cleverest prompts. They'll be the firms that engineered for the four-hundredth run — on the Tuesday everything went wrong — and shipped it anyway.

Building agentic workflows your firm can actually trust? We would welcome a 20–30 minute conversation about where agentic AI could create real value for your firm, and what a production-grade architecture for it looks like. No commitment — just a practical conversation with the people who build these systems.

West Stack — weststack.io