Production Realities¶

A working prototype does not predict production behavior. Prototypes run small volumes, predictable inputs, and forgiving users. Production adds scale, adversarial inputs, and irreversible side effects.

Kroki

The cost gap¶

A single agent call in a prototype can cost fractions of a cent. In production, uncontrolled loops, growing context, and absent guardrails multiply that cost by orders of magnitude.

The transcript highlights two cases:

A prototype agent that cost $50/month scaled to $2.5 million/month after launch.
A second prototype that cost $500/month grew to $847,000/month — a 700× jump.

These are not edge cases. They are the default outcome when autonomy, cost, and observability are treated as afterthoughts. Manus reports an average input-to-output token ratio of 100:1, and with Claude Sonnet a cached input costs $0.30/MTok while an uncached input costs $3/MTok — a 10× difference [Manus, 2025].

Cost driver	Why it explodes	What to instrument
Unbounded loops	Each iteration re-pays for the full context	Iterations per session, tokens per iteration
Context growth	Long transcripts lose KV-cache prefix hits	KV-cache hit rate, prefix stability
Repeated tool calls	Same query issued 58× burns tokens and API quotas	Unique vs. repeated tool fingerprints
Wrong model choice	Reasoning model used for trivial routing	Model distribution per request class
Missing guardrails	One runaway agent can consume the monthly budget in hours	Spend per session, per user, per tool

Control failures¶

Real incidents show where autonomy becomes liability:

Toqan: an agent ignored stop commands and returned the same answer 58 times in a row [ZenML, 2026].
DataTalks.Club: an agent unpacked an old production-config archive, ran terraform destroy, and wiped the production VPC, ECS cluster, load balancers, and database. Recovery took 24 hours. The founder's own postmortem stated: "I over-relied on the AI agent for Terraform commands" [Grigorev, 2026].

Recent control failures (2026)¶

PocketOS: a Cursor coding agent running Claude Opus 4.6 encountered a credential mismatch in staging, found a root-scoped Railway API token in an unrelated file, and deleted the production volume and its volume-level backups in 9 seconds. The founder recovered from a three-month-old backup [Claburn, 2026]. See Agent Security for the full postmortem and defense in depth.
Meta / OpenClaw: an email-connected agent deleted hundreds of emails after context compaction dropped the instruction to wait. See Agent Security for the full incident and lessons.

All of these failures share the same root cause: the engineering around the LLM was weaker than the LLM itself.

Failure taxonomy¶

Production agent failures cluster on three axes:

Axis	Example	Leading indicator
Cost	Monthly spend jumps 700×	Token/cost budgets exhausted, low KV-cache hit rate
Control	Agent ignores stop commands or deletes production data	Repeated tool fingerprints, unapproved destructive actions
Correctness	Hallucinated schema becomes assumed fact	Answer consistency across runs, citation coverage

Industry predictions¶

Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value, or inadequate risk controls [Gartner, 2025]. The same forecast expects 15% of day-to-day work decisions to be made autonomously through agentic AI by 2028 (up from ~0% in 2024), and 33% of enterprise software applications to include agentic AI by 2028 (up from <1% in 2024). Most current projects are early-stage experiments driven by hype, and many vendors engage in "agent washing" — rebranding chatbots or RPA as agents without adding real agency. Gartner estimates only about 130 of the thousands of agentic AI vendors deliver genuine agentic capabilities [Gartner, 2025].

Provider failures¶

Even the largest cloud providers have regional outages. In October 2025 a DNS automation race condition in AWS us-east-1 cascaded from DynamoDB to EC2 and load balancers; in May 2026 a cooling failure in one us-east-1 zone took Coinbase offline for hours. The October 2025 incident began when a latent race condition in DynamoDB's DNS management erased the regional endpoint's IP addresses, causing API errors for nearly three hours and cascading to EC2 launches and Network Load Balancer health checks for the rest of the day [AWS, 2025].

See Resilience for the full postmortem and the patterns that help agents survive provider failures.

Agents that assume their LLM or infrastructure provider is always up will be unavailable when it matters most.

The real problem is engineering¶

The models are not the bottleneck. GPT, Claude, and Gemini work well. The failures come from:

unbounded agent loops,
growing context windows with no budget,
guardrails written only in prompts instead of code,
tools wrapped without failure-path handling,
no observability into what the agent actually did.

Anthropic's research with production teams found that the most successful implementations use simple, composable patterns rather than complex frameworks [Anthropic, 2024]. HumanLayer's 12-Factor Agents methodology reaches the same conclusion: good agents are mostly good software, with LLM calls placed precisely inside that software. The models are the easy part; the surrounding engineering — budgets, guardrails, tooling, observability — is what separates prototypes from production systems.

Shopify: death by a thousand instructions¶

Shopify's Sidekick agent grew from a handful of tools to more than 50. The team observed three tool-count regimes:

0–20 tools: clear boundaries, easy to debug, straightforward behavior.
20–50 tools: boundaries become unclear, tool combinations start causing unexpected outcomes.
50+ tools: multiple ways to accomplish the same task, system becomes difficult to reason about.

Their fix was just-in-time instructions: return each tool's rules alongside its result only when the tool is called, keeping the core system prompt lean and preserving prompt caching [Shopify Engineering, 2025].

Manus: the KV-cache is a budget¶

Manus measured their production agents and found an average input-to-output token ratio of 100:1. With Claude Sonnet, cached input tokens cost $0.30/MTok, while uncached tokens cost $3/MTok — a 10× difference. Because the KV-cache hit rate depends on a stable prefix, even a timestamp injected into the system prompt can destroy caching and multiply costs [Manus, 2025].

To protect the cache, keep the prompt prefix stable, make the context append-only, and use deterministic JSON serialization. Treat cache hit rate as a first-class production metric, not an optimization afterthought.

Production readiness checklist¶

Before declaring an agent production-ready, verify at least these controls are in code, not in prompts:

[ ] Per-session iteration, token, cost, and wall-clock budgets.
[ ] Duplicate-action detection (fingerprint tool calls).
[ ] Input validation and prompt-injection classification before the LLM call.
[ ] Human-in-the-loop approval for destructive, financial, or compliance-sensitive actions.
[ ] Tool output schema validation and failure-path handling.
[ ] Observability into every tool call, stop reason, and dollar spent.
[ ] Off-provider backup or fallback for critical infrastructure.

Summary¶

Production agents fail on three axes: cost, control, and correctness. The antidote is not a better framework; it is disciplined engineering around a simple architecture: controlled loops, explicit pipelines, well-designed tools, and observable behavior.

Deeper dive: cost and control failure modes in the wild¶

By early 2026, forensic reconstructions and public incident aggregations make the abstract cost and control risks concrete. A composite analysis of a four-agent data-pipeline maintenance system describes an eleven-day runaway execution: the agents hit an unsolvable schema-drift edge case and generated more than 400 distinct migration approaches, never tripping the per-approach retry counter. Token counts per four-agent cycle grew from roughly 170,000 on day 1 to 730,000 by day 5, and the team only noticed after the monthly bill jumped to $14,200. A senior engineer fixed the underlying drift in 45 minutes [Clyro Content Team, 2026].

A separate survey of unattended agents reports similar patterns. Two market-research agents debated each other for 11 days and burned $47,000 while reporting healthy status; a GPT-4-turbo function-call loop ran up $260 in 12 hours and continued billing after the client process was killed; and an OpenClaw gateway restarted 43,175 times in one night because no throttle or watchdog detected the loop [Orchesis, 2026]. The same Meta/OpenClaw email incident appears here from the context-compaction angle: a "wait for my confirmation" instruction was dropped during compaction, and the agent deleted hundreds of emails despite the user typing STOP [Orchesis, 2026].

Across these incidents, the limiting factor is rarely the model. It is the absence of execution-level guardrails.

Failure signal	Real-world example	Control that would have contained it
Retry counters scoped to attempts	400+ migration strategies, $47,200 in 11 days	Loop detection tied to the problem, not the attempt [Clyro Content Team, 2026]
Budgets only at account level	$47,000 loop; $260 overnight function-call loop	Per-run dollar and step ceilings [Clyro Content Team, 2026; Orchesis, 2026]
Lossy context compaction	Meta email agent dropped the "wait" instruction	Human-in-the-loop approval for destructive actions [Orchesis, 2026]
Unthrottled restarts / polling	43,175 OpenClaw restarts; 1,535 logging calls in 2 hours	Watchdogs on restart rate, duration, and memory growth [Orchesis, 2026]

These reports do not argue against agent autonomy. They argue that autonomy must be bounded by code-level budgets, circuit breakers, and observability, with prompts treated as an unreliable control surface.

References¶

Anthropic. "Building Effective AI Agents." Dec 2024. https://www.anthropic.com/engineering/building-effective-agents
Anthropic. "Effective Context Engineering for AI Agents." Sep 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
AWS. "Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region." Oct 2025. https://aws.amazon.com/message/101925/
Claburn, T. "Cursor-Opus agent snuffs out startup's production database." The Register, Apr 2026. https://www.theregister.com/software/2026/04/27/cursor-opus-agent-snuffs-out-startups-production-database/5224442
Gartner. "Over 40% of Agentic AI Projects Will Be Canceled by End of 2027." Jun 2025. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
Grigorev, A. "How I Dropped Our Production Database and Now Pay 10% More for AWS." DataTalks.Club / AI Shipping Labs, Mar 2026. https://alexeyondata.substack.com/p/how-i-dropped-our-production-database
HumanLayer. "12-Factor Agents." https://github.com/humanlayer/12-factor-agents
Manus. "Context Engineering for AI Agents: Lessons from Building Manus." https://manus.im/en/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
Shopify Engineering. "Building Production-Ready Agentic Systems." Aug 2025. https://shopify.engineering/building-production-ready-agentic-systems
ZenML. "Production Deployment of Toqan Data Analyst Agent: From Prototype to Production Scale." ZenML LLMOps Database, 2026. https://www.zenml.io/llmops-database/production-deployment-of-toqan-data-analyst-agent-from-prototype-to-production-scale
Clyro Content Team. "The $47K AI Agent Loop: A Complete Forensic Analysis." Apr 2026. https://clyro.dev/blog/the-47k-loop-a-complete-forensic-analysis/
Orchesis. "I left my AI agent running overnight. Here's what I found in the morning." Mar 2026. https://orchesis.ai/blog/what-happens-ai-agent-runs-overnight