Skip to content

Concepts

This section turns the video transcript into a structured, engineering-focused reference for building production-grade agentic systems. It is organized around six anchors:

  1. Production realities — why a prototype that costs $50/month can turn into a $2.5M/month production bill.
  2. Workflow vs agent — a seven-level ladder that starts with a single LLM call and ends with multi-agent systems, with clear stop rules.
  3. A composable pipeline — how a request is validated, enriched, assembled into a context window, executed, and observed.
  4. Intelligence infrastructure — model routing, memory, retrieval, and context budgets; the intelligence of an agent lives in what surrounds the model, not in the model alone.
  5. Security and resilience — defense in depth for agents, provider failure modes, and the engineering harness that keeps autonomy safe.
  6. Economics and observability — why falling token prices do not mean lower bills, and which metrics reveal what dashboards hide.

All diagrams are stored as plain-text sources under docs/diagrams/ and rendered through the Kroki service configured in the project.

Pages

  • Production Realities — incidents, cost explosions, and industry predictions.
  • Agent vs Workflow — the seven-level pattern ladder.
  • Pipeline — the fail-fast request pipeline and context assembly.
  • Agent Loop — ReAct, guardrails, and recovery strategies.
  • Tools and MCP — tool design, MCP, and the happy-path trap.
  • Observability — tracing, metrics, and agent-specific signals.
  • Agent Security — prompt injection, excessive agency, confused deputy, and defense in depth.
  • Resilience — fail-closed, fail-open, graceful degradation, and surviving provider outages.
  • Harness Engineering — the runtime, tooling, and controls that surround the model.
  • Token Economics — Jevons paradox, falling prices, and rising enterprise AI bills.
  • Model Selection — choosing between API, self-host, and hybrid, and evaluating on your own data.
  • Smart Routing — choosing the right model for each request.
  • RAG vs Long Context — when to retrieve and when to stuff the window.
  • Memory Architecture — the four memory types from the CoALA framework.
  • Claude Code Memory — production memory architecture from the leaked build analysis.
  • Context Budget — token budgets and thinking strategy.
  • References — all external sources used in this section.

Summary

The intelligence of an agent is not in the model. It is in the infrastructure around the model: choosing the right deployment and model portfolio, routing requests to the cheapest capable model, choosing between retrieval and long context, building the four memory systems from CoALA, designing production memory pipelines like Claude Code's, and enforcing a strict context budget with a deliberate thinking strategy.

These five components multiply. A cheap model with great routing and a focused context window can outperform an expensive model fed an unbounded prompt. A memory system that records and retrieves lessons prevents the agent from repeating past failures. And observability over every routing, retrieval, and compaction decision is what turns cost and quality from mysteries into measured engineering trade-offs.

Part 3 moves from architecture to production: security keeps autonomy from becoming liability, resilience decides what the system does when a provider fails, harness engineering treats the model as one component inside a larger runtime, token economics explains why cheaper tokens can mean larger bills, and observability exposes the hidden loops and costs that dashboards miss.