Memory Architecture¶

Saying an agent has "memory" is as vague as saying a computer has "storage." A computer has registers, caches, RAM, SSDs, and archives; each has a different lifetime and purpose. Agents have a similar taxonomy, formalized by Princeton researchers as CoALA (Cognitive Architectures for Language Agents).

CoALA organizes language agents around memory, action, and decision-making. The memory layer alone has four distinct types, each with its own lifecycle and failure mode.

Kroki

The evolution of agent memory¶

A 2026 survey organizes LLM-agent memory research into three evolutionary stages [Luo et al., 2026]:

Stage	What it does	CoALA mapping
Storage	Faithfully records interaction trajectories	Episodic traces
Reflection	Actively refines, deduplicates, and scores stored records	Semantic and episodic updates
Experience	Abstracts cross-trajectory patterns into reusable skills and strategies	Procedural and higher-level semantic memory

Most production systems are still in Storage and early Reflection. Experience-stage systems use active exploration and cross-trajectory abstraction to turn scattered episodes into stable priors.

1. Procedural memory: how to act¶

Procedural memory is the agent's job description. It contains the system prompt, tool descriptions, output formats, and behavioral guardrails. It is explicit, designer-authored, and stable within a session.

Scope: session-wide.
Updates: intentional, between sessions, not on the fly.
Risk: if it grows too large, it crowds out everything else in the context window. Tools and rules should be modular and loaded just-in-time when possible.

2. Semantic memory: facts the agent knows¶

Semantic memory stores world knowledge, user preferences, project conventions, and domain facts. This is where RAG and structured knowledge bases live. Unlike procedural memory, it is expected to change as new information arrives.

For the barista:

Kostya likes cappuccino.
The payment microservice is written in Go with a clean architecture.
The return policy allows refunds within 14 days.

For a coding agent:

The team uses PostgreSQL.
This module follows hexagonal architecture.
This API is deprecated.

Semantic memory is a living reference book, not a static dump.

3. Episodic memory: what happened and what it taught us¶

Episodic memory is the most underused type in production systems. It stores concrete episodes with context, action, result, and lesson. It turns past failures and successes into warnings and recommendations.

Example episodes:

Context: Last Tuesday, a customer with a nut allergy ordered a drink. Action: The agent served a latte with almond milk. Result: Incident, complaint. Lesson: Always double-check allergy tags before confirming a dairy substitute.
Context: The agent refactored a module without interfaces. Action: Rollback. Result: Broken build. Lesson: Refactor through interfaces; never break the public contract.

Before taking an action, the agent queries episodic memory: "Have similar actions gone wrong before?" If a negative episode matches, the lesson is added to the working context as a warning. After the action, the agent records the result. This creates two extra calls per action, so the store needs forgetting, weight decay, or eviction to avoid infinite growth.

Recent work argues episodic memory is the missing piece for long-term agents and identifies five properties that distinguish it: long-term storage, explicit reasoning, single-shot learning, instance-specific content, and contextualized retrieval [Pink et al., 2025]. A 2026 study adds a warning: continuously asking an LLM to consolidate past trajectories into textual lessons can make memory less useful over time. On ARC-AGI problems, GPT-5.4 failed 54% of tasks it had previously solved after consolidation, while an episodic-only control that kept raw trajectories retained most of its accuracy [Zhang et al., 2026]. The practical lesson: keep raw episodes as first-class evidence, and gate consolidation with error checks or human review.

4. Working memory: what matters right now¶

Working memory is the current task context. Five people need a retro in an hour, a hotfix just shipped, the temperature is -15°C. This information is temporary; it is discarded when the task ends.

Working memory is closest to the LLM's own context window, but it is not identical. It is the curated, assembled content that the agent decides is relevant to this turn.

Lifecycle summary¶

Memory type	Changes	Lifetime	Example
Procedural	Intentionally, between sessions	Session	System prompt, tool descriptions
Semantic	Continuously as data arrives	Long-term	User preferences, project facts
Episodic	After each action, with forgetting	Long-term, decaying	Past incidents and lessons
Working	Every turn	Task	Current order, current task context

Why it matters¶

Most production agents are good at procedural and semantic memory, but skip episodic memory. The result is an agent that knows the rules and the facts, but repeats the same mistakes. Building episodic memory means adding:

A structured record format: context, action, result, lesson.
A pre-action retrieval step to find relevant past episodes.
A post-action write step to record new lessons.
A forgetting or weighting policy to keep the store usable.

Failure modes and metrics¶

Failure	Symptom	Metric
Memory bloat / unbounded growth	Latency and cost rise every turn	Store size, tokens retrieved per turn
Stale or contradictory facts	Agent uses outdated convention	Contradiction rate vs. source of truth
Over-consolidation	Lessons become generic or wrong	Task accuracy with vs. without memory
Retrieval miss	Relevant past episode not surfaced	Recall@k for held-out episodes
Wrong-memory injection	Agent acts on irrelevant old episode	Precision@k, user correction rate

Treat memory quality as a retrieval problem, not just a storage problem.

Design principles¶

Match storage to lifetime. Do not put temporary task state into long-term memory.
Write lessons, not logs. Episodic memory is insights, not raw transcript.
Forget deliberately. Without eviction, episodic memory becomes noise.
Gate consolidation. Keep raw episodes as evidence; verify distilled lessons before writing them back.

Deeper dive: production memory implementations¶

By 2026 the production ecosystem has converged on a tiered memory design that looks more like an operating system than a single vector index. The dominant pattern is a small always-in-context working tier, a retrievable episodic/semantic layer, and a durable long-term store; the agent itself moves data between tiers through explicit tool calls rather than relying on context injection [Zylos Research, 2026]. Vector stores are good at fuzzy semantic recall but structurally blind to relationships, while knowledge graphs handle relational and temporal reasoning precisely but require ontology maintenance; production systems increasingly layer both and let the LLM decide what to retrieve, write, and forget [Zylos Research, 2026].

The four CoALA memory types show up in concrete but different ways across frameworks. Letta exposes core_memory_replace, archival_memory_search, and archival_memory_insert calls that page working, recall, and archival tiers in and out of the context window [Zylos Research, 2026]. Mem0 extracts named entities and relationships, stores them as a graph linked to vector embeddings, and runs a conflict-detection pass before updating facts, which it claims improves LOCOMO accuracy while cutting latency and token use [Zylos Research, 2026]. Zep's Graphiti engine annotates every fact with both event time and ingestion time, so retroactive corrections such as an address change can coexist with prior beliefs without information loss [Zylos Research, 2026]. Procedural memory is still the least mature tier: the simplest form is a project-level markdown file such as CLAUDE.md or AGENTS.md injected into the system prompt, while LangMem lets an agent call update_system_prompt to rewrite its own instructions at runtime [Zylos Research, 2026; JobsByCulture, 2026].

A tiered design is not optional even with million-token context windows. Raw context stuffing is expensive, still suffers from "lost in the middle" attention degradation, and cannot keep up with sessions that run for weeks or months [JobsByCulture, 2026]. Teams report roughly a 4× cost reduction when moving from full-context stuffing to tiered retrieval [JobsByCulture, 2026]. The more subtle failure is memory hygiene: without TTLs, decay scores, deduplication, and conflict resolution, long-term stores degrade into contradictory noise within weeks [JobsByCulture, 2026]. Benchmarks such as LoCoMo and LongMemEval now measure episodic recall, but they still miss procedural memory quality, cross-agent consistency, and resistance to memory poisoning [Zylos Research, 2026].

Framework	Best for	Memory model	Hosting
LangMem	LangGraph teams; background extraction	Episodic + semantic + procedural	Self-host
Letta	Long-running agents needing OS-style control	Core + recall + archival; LLM-managed paging	Self-host / cloud
Mem0	Fast managed integration; user/session/agent scope	Semantic facts + graph extraction linked to vectors	Managed / self-host
Zep	Temporal reasoning and knowledge-graph queries	Bitemporal knowledge graph (Graphiti)	Cloud / self-host

References¶

Wang, S., et al. "Cognitive Architectures for Language Agents." arXiv, 2023. https://arxiv.org/abs/2309.02427
Pink, M., et al. "Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents." arXiv, 2025. https://arxiv.org/abs/2502.06975
Luo, J., et al. "From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms." arXiv, 2026. https://arxiv.org/abs/2605.06716
Zhang, D., et al. "Useful Memories Become Faulty When Continuously Updated by LLMs." arXiv, 2026. https://arxiv.org/abs/2605.12978
Zylos Research. "AI Agent Memory Architectures: From Context Windows to Persistent Knowledge." Zylos Research, 2026. https://zylos.ai/research/2026-04-05-ai-agent-memory-architectures-persistent-knowledge/
JobsByCulture. "AI Agent Memory Systems: A 2026 Engineering Guide (Letta, LangMem, Mem0, Zep)." JobsByCulture, 2026. https://jobsbyculture.com/blog/ai-agent-memory-systems-guide-2026