Why AI agents fail in production

A few weeks ago I was talking to a team at a large financial company that has spent the last year building AI into one of its core operational workflows.

They had made real progress. They’d broken the process into individual checks, used LLMs to automate parts of the review, and were already seeing work move faster. Now they wanted to take the next step and let the system make some decisions without a person reviewing every case.

That’s where they hit a problem.

Some of their checks were easy to make deterministic. If you need to verify a fact against a database, you can look it up and compare the result. But other checks required reading an unstructured document and deciding whether it satisfied a policy. An LLM could do that well, but n

kkkot with the consistency they needed to let it make the final decision.

So they were going back through the entire workflow, check by check, asking where they could replace an LLM with deterministic logic and where they actually needed the judgment of an agent.

Thanks for reading The Operator's Guide to AI! Subscribe for free to receive new posts and support my work.

Subscribed

I’ve had versions of that conversation across compliance, underwriting, due diligence, and research workflows over the past three years.

The prototype is rarely the hard part. Problems appear when teams run the same workflow hundreds of times and need consistent results, traceable decisions, and predictable costs.

A Deloitte study found that 11% of organizations were running agents in production, while another 14% had solutions ready to deploy. Model performance improved over the same period. On WebArena, which tests agents on multi-step web tasks, the best-performing agent rose from 14.4% in 2024 to 71.6% in 2026.

You’d expect those improvements to translate into more agents making it into production. They haven’t.

We examined two independent sources of production evidence. Clyro collected 591 documented agent failures, while Presenc AI analyzed public evaluations and deployment instrumentation from more than 60 enterprise agent customers. In Clyro’s dataset, 88% of failures involved infrastructure around the model. Hallucinations and other model-quality failures accounted for around 10%.

That finding matches our experience pretty closely.

One way to understand the problem is through some simple math. Say an agent has to complete a 20-step workflow and is 90% accurate at each step. A 90% success rate sounds pretty good in isolation. Across all 20 steps, though, the probability of completing the entire workflow successfully falls to 12.2%.

At 50 steps, the probability falls to 0.5%.

Production workflows also contain dependent errors. If an agent makes a bad tool call at step three, that result enters the context for step four. The agent makes its next decision using bad information, then carries that decision into the following step. It can execute a coherent plan built on a mistake it made ten minutes earlier.

When someone shows me an agent performing an impressive task, I want to know what happens on the hundredth run, then the thousandth. I want to know how it handles an API timeout halfway through a task, contradictory sources, or a context window crowded enough to obscure an instruction from the begin