Why memory is the real product surface

Every team I talk to ships an agent and then immediately runs into the same wall: the model is smart, but the system is forgetful. The wall is not the model. It is the absence of a clear memory taxonomy. Once you name the layers, the architecture writes itself.

The four layers

1. Working memory (the prompt window)

This is what the model can "see" right now. It is expensive, volatile, and where most teams overspend. Treat the context window like a stage, not a warehouse. The only things that belong here are the current task, the last few turns, and the artifacts the model needs to manipulate this step.

2. Episodic memory (the session log)

Everything the user said and the agent did during this session. Cheap to store, easy to summarize. Episodic memory is what lets an agent say "earlier you asked me to…" without re-reading the whole transcript. The trick is rolling summarization: compress old turns into structured facts, keep recent turns verbatim.

3. Semantic memory (RAG and vector stores)

The library the agent can look things up in. Docs, tickets, prior decisions, knowledge base. This is the layer most teams build first and most teams over-rely on. Vector search is a retrieval mechanism, not a reasoning mechanism. If your agent feels dumb, the answer is rarely "more chunks."

4. Persistent world model (the user and workspace state)

Who the user is, what they care about, what they have committed to, what state their workspace is in. This is the layer that turns a chatbot into a coworker. It lives in your database, not your vector store, and it should be queryable with plain SQL.

The boundaries that matter

  • Working vs. episodic: the moment a turn is no longer being referenced, it should leave the prompt and enter the session log.
  • Episodic vs. semantic: episodic is "what happened with this user"; semantic is "what is true in the world." Do not mix them in the same index.
  • Semantic vs. world model: if the answer changes per user, it does not belong in the shared vector store. Put it in the world model.

A simple rule

Before you add another retrieval call, ask which of the four layers the missing information belongs to. If you cannot answer, you do not have a memory problem. You have a taxonomy problem.

— Aadhar