Agent Memory Systems
A language model on its own has no memory whatsoever between two separate calls. Whatever you see as "memory" in an agent is actually a layer engineered around the model.
Why Does an Agent Need Memory?
Without memory, every interaction with an agent starts from zero: the user has to reintroduce themselves every time, restate their preferences, and the agent learns nothing from prior interactions. Memory solves this — but comes in several types, each solving a different problem.
Short-term Memory
The conversation's current context window — the message history of this particular chat. The simplest kind of memory, but it disappears when the conversation ends or the context window fills up.
Long-term Memory
Information that needs to persist across different sessions (user preferences, purchase history). Usually stored in a vector database, and the relevant parts are retrieved at the start of each new conversation — using exactly the same mechanism as RAG.
Episodic Memory
A specific record of past events: "the user placed order X on 2026-03-12 and later cancelled it." Unlike semantic memory, which holds "general facts," episodic memory records "specific events" with time details.
Semantic Memory
General, stable facts about the world or the user: "the user is interested in sports products," "the user's shoe size is 9." This type of memory is usually derived by consolidating several events or repeated interactions.
Procedural Memory
Knowledge of "how to do" a task — for example, the correct sequence of tool calls to complete an order. This type of memory is often embedded as rules, a template, or even the system prompt/code itself, rather than a simple data record.
How to Implement It
| Memory type | Common storage location |
|---|---|
| Short-term | A message array in application memory (RAM) |
| Long-term / Semantic | A vector database |
| Episodic | A relational database or a structured log |
| Procedural | The system prompt, a rules file, or code |
Forgetting & Consolidation
Memory that never deletes anything gradually turns into a disorganized, costly hoard — both in terms of storage space and retrieval quality (the more irrelevant records there are, the noisier semantic retrieval gets). This is why mature memory systems usually have a management layer as well:
- Consolidation — several small, related events (episodic memory) get gradually summarized into one more general fact (semantic memory); for example, five sneaker purchases turn into the record "the user is interested in athletic wear."
- Decay — older, less-used records get a lower score in retrieval ranking so fresher information takes priority.
- Explicit deletion — a user must be able to clear part of their own memory (a common privacy requirement); this means memory needs to be designed from the start to support selective deletion, not just additions.
FAQ
Does every agent need all these types of memory?
No. A simple Q&A agent might only need short-term memory; a personalized shopping assistant needs a combination of all four types.
What's the difference between long-term memory and RAG?
RAG is a retrieval technique; long-term memory is one application of that same technique — you run RAG over "an organization's general knowledge," and long-term memory over "one user's specific history."