Agent Loop¶
The agent loop is what makes an agent an agent: the LLM decides whether to call a tool, the tool result is appended to context, and the LLM is called again. The loop repeats until the LLM decides to answer.
It is also the most dangerous component.
ReAct loop¶
The basic flow:
- LLM receives the current context.
- LLM decides: call a tool or provide an answer.
- If it calls a tool, the request is validated and executed.
- The result is appended to the context.
- Repeat.
Why it fails¶
- Loss of control: the agent ignores stop conditions and repeats the same action. Toqan's data-analyst agent returned the same answer 58–59 times in a row before hard limits were added [ZenML, 2026].
- Cost: every iteration pays for the full context. Manus reports an average 100:1 input-to-output token ratio [Manus, 2025].
- Correctness: a hallucination in step 3 becomes assumed fact by step 4 and the basis for a decision by step 5.
- Stagnation: the model calls the same tool with the same arguments and fails to integrate the result, burning tokens without progress. Harbor Analytics cut runaway loops 91% by adding iteration caps, stagnation fingerprints, and a token-budget governor [Solana Garden, 2026].
- Compounding errors: a bad tool call in one turn biases every later turn.
Guardrails¶
Guardrails must be hard limits in code, not suggestions in a prompt.
| Guardrail | Purpose | Example limit |
|---|---|---|
| Iteration limit | Maximum loop cycles | 5 for support, 25 for code agents |
| Token budget | Maximum total spend per request | sum of all LLM calls |
| Cost budget | Maximum dollars per user, ticket, or organization | essential for multi-tenant SaaS |
| Wall-clock timeout | Absolute elapsed time including tool I/O | prevents a slow SQL query from blocking forever |
| Per-tool call limit | Prevent repeated identical tool calls | weather API max 3 times |
| Action fingerprint | Hash of (tool_name, canonical_args) |
halt after 2 repeats |
| Anomaly detection | Pause on unusual parameters | 500 espressos or sum > daily budget |
| Human-in-the-loop | Require approval for irreversible actions | payment, delete, deploy |
When a guardrail fires, tell the LLM exactly what happened and what to do next. "Error" is not enough. "Tool get_weather called 4 times, max is 3; you have enough data to answer" is enough.
Termination taxonomy¶
Mature runtimes combine several independent stop signals. Treat success predicates as an OR and guardrails as an AND [Solana Garden, 2026].
| Category | Stop signal | When to use |
|---|---|---|
| Hard budgets | Max iterations | Always; typical support 5–15, research 20–40 |
| Token / cost budget | Multi-tenant SaaS, long-horizon tasks | |
| Wall-clock timeout | Any tool with variable latency | |
| Goal predicates | Structured finish tool |
Require the model to call submit_answer with valid schema |
| External verifier | A rules engine confirms required slots are filled | |
| Stagnation | Action fingerprint | Same (tool, args) twice in a row |
| Observation similarity | Cosine similarity > 0.98 for three consecutive results | |
| State oscillation | A → B → A alternation with no new facts | |
| Failure paths | Tool error budget | Stop after K consecutive tool failures |
| Confidence floor | Self-reported confidence below threshold for two turns | |
| Human handoff | Package scratchpad, trace, and partial answer for review |
Human involvement levels¶
- Human in the loop: the human makes a decision before the action continues.
- Human on the loop: the human observes and can intervene.
- Human out of the loop: the system is fully autonomous.
Use the first for irreversible or high-risk actions. AWS Well-Architected recommends risk-tiered approval: pause agents only for decisions where human judgment changes the outcome, give reviewers enough context to decide, and log every approval with timestamp and reviewer identity [AWS, 2025]. Routing every action through a human creates rubber-stamp approvals; routing none creates unbounded autonomy [AWS, 2025].
Recovery strategies¶
When the loop is stopped, the agent should not return silence. Options:
- Partial result: return what was completed and ask the user to fill the rest.
- Escalation: hand off to a human operator.
- Retry with simplified context: use a different model or smaller context.
- Degraded single-shot fallback: one final non-tool completion using compressed context when budgets exhaust [Solana Garden, 2026].
What to instrument¶
You cannot tune what you do not measure. Track:
- Iteration count and which stop condition fired first.
- Token and cost spend per session, per user, per tool.
- KV-cache hit rate (prefix stability).
- Duplicate tool-call fingerprints.
- Tool error rate and recovery rate.
- Human escalation rate and reason.
Deeper dive: production loop patterns¶
Recent field reports confirm that the failures above are not edge cases. In mid-2025 a Claude Code recursion loop reportedly burned between $16,000 and $50,000 in five hours, and a four-agent LangChain loop ran for eleven days and cost $47,000 before anyone noticed; in both cases the agents were functioning correctly, but nobody had defined when to stop [Nwaneri, 2026]. The common factor is not model quality but missing exit discipline.
Production-grade loops separate three concerns: defining done, enforcing limits, and preserving evidence. A spec written before the loop starts forces a one-sentence answer to "what does done look like" and becomes the benchmark the runtime can actually check against [Nwaneri, 2026]. A circuit breaker is implemented as an exception, not a return code, so a breached turn or token limit halts the loop before the next LLM call is paid for [Nwaneri, 2026]. Between those boundaries, an append-only ledger records every turn, token delta, and pass/fail state so the session remains auditable and downstream systems receive nothing until a human attests the result [Nwaneri, 2026].
Recovery also needs to be designed in, not bolted on. Durable execution treats the loop body as a workflow: each decision and result is written to an append-only event history so a crashed process can replay to the same point without redoing side effects [Hill, 2026]. Because model calls are non-deterministic, only their results are replayed, never re-executed. Before adding retries, actions must be idempotent with a deduplication token; otherwise a crash-and-resume becomes a double-execution generator with good intentions [Hill, 2026]. Retries themselves should be bounded, backed off, and circuit-broken, and runs that never converge should be quarantined rather than left to exhaust budgets [Hill, 2026].
| Production pattern | What it buys you | Failure it prevents |
|---|---|---|
| Pre-loop spec writer | A concrete, one-sentence done condition | Vague goals the model cannot satisfy |
| Pre-flight circuit breaker | Halt before the next paid LLM call | Burning tokens after limits are already exceeded |
| Append-only ledger | Immutable, per-turn audit trail | Silent drift and unverifiable outputs |
| Durable execution / checkpoint | Resume from the last good step | Losing progress on crash and redoing side effects |
| Idempotency tokens | Safe retries and replays | Double-charges, double-writes, duplicate side effects |
| Quarantine for non-convergers | Isolation of pathological runs | Budget exhaustion on inputs the model cannot resolve |
References¶
- Anthropic. "Building Effective AI Agents." Dec 2024. https://www.anthropic.com/engineering/building-effective-agents
- AWS. "AGENTSEC04-BP02 Human-in-the-loop for critical decisions." AWS Well-Architected Agentic AI Lens, 2025. https://docs.aws.amazon.com/wellarchitected/latest/agentic-ai-lens/agentsec04-bp02.html
- Manus. "Context Engineering for AI Agents: Lessons from Building Manus." https://manus.im/en/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus
- Solana Garden. "LLM Agent Loop Termination Explained: Stopping Criteria, Budget Caps and Production Guardrails." Jun 2026. https://solana.garden/guides/llm-agent-loop-termination-explained/
- Zalesov, A. "A Practical Memo on Building LLM Agents." May 2026. https://medium.com/@zallesov/a-practical-memo-on-building-llm-agents-cc5179cb2e50
- ZenML. "Production Deployment of Toqan Data Analyst Agent: From Prototype to Production Scale." ZenML LLMOps Database, 2026. https://www.zenml.io/llmops-database/production-deployment-of-toqan-data-analyst-agent-from-prototype-to-production-scale
- Hill, Brenn. "Failure Recovery for Agent Loops: Retries, Rollback, and Resuming a Crashed Run." LoopRails, Jun 2026. https://looprails.dev/article-failure-recovery-agent-loops.html
- Nwaneri, Daniel. "How to Build a Production-Safe Agent Loop: From Exit Conditions to Audit Trails." freeCodeCamp, Jun 2026. https://www.freecodecamp.org/news/how-to-build-a-production-safe-agent-loop-from-exit-conditions-to-audit-trails/