AI agent memory is how an agent keeps track of information across a task instead of treating every step like the first one. Without memory, an agent would forget what tool it just called, what the user already told it, and what it already tried.
There are two broad kinds: short-term memory, which holds what's relevant to the current task, and long-term memory, which persists across separate sessions or conversations entirely.
Getting memory right is one of the harder parts of building a real agent — too little, and the agent repeats itself or contradicts earlier steps; too much, and it gets slow, expensive, and confused by irrelevant history.
Why agents need memory at all
A language model on its own has no memory between calls — each request is independent unless you explicitly include the relevant history. An agent running a multi-step loop needs to know what it already tried, what results it got, and what the user's actual goal was several steps back. Memory is the mechanism that carries that information forward.
Short-term memory: what's happening right now
Short-term memory (sometimes called working memory) holds the context relevant to the current task: the conversation so far, recent tool results, and the plan the agent is currently executing.
- Lives inside the current session or task.
- Usually implemented as the conversation history sent with each model call.
- Disappears once the session ends, unless explicitly saved somewhere else.
The practical limit on short-term memory is the model's context window — the amount of text it can consider at once. Long tasks with lots of tool calls can fill that window fast, which is why agents often summarize or trim older steps rather than keeping everything.
Long-term memory: what persists across sessions
Long-term memory is information the agent can recall in a future session — a user's preferences, facts learned in a past conversation, or outcomes from earlier tasks.
- Usually stored outside the model itself, in a database or vector store.
- Retrieved selectively when relevant, not loaded all at once.
- Lets an agent "remember you" across separate conversations, days or weeks apart.
Short-term vs long-term memory
| Short-term memory | Long-term memory | |
|---|---|---|
| Scope | Current task or session | Across sessions, over time |
| Typical storage | Conversation history, in context | External database or vector store |
| Goal | Stay coherent through the current job | Recall relevant facts from the past |
| Example | Remembering a tool result from two steps ago | Remembering a user prefers metric units, from last month |
How agents store and retrieve long-term memory
Most long-term memory systems follow a pattern similar to retrieval-augmented generation (RAG): information is stored as embeddings in a vector database, then the agent searches for the most relevant memories given the current situation, rather than loading its entire history every time.
Common approaches include:
- Episodic memory — logging specific past events or interactions, retrieved when similar situations come up again.
- Semantic memory — storing general facts or preferences, independent of any single conversation.
- Summarized memory — periodically compressing old conversation history into shorter summaries, so key facts survive without keeping every raw message.
What can go wrong with agent memory
- Context overflow — cramming too much raw history into the context window, crowding out the actual task.
- Stale memory — an agent recalling an outdated fact (a preference the user changed) as if it were still true.
- Irrelevant retrieval — pulling long-term memories that sound similar but don't actually apply to the current situation.
- No memory at all — the simplest failure: an agent that repeats questions it already asked, because nothing was carried forward.
Memory isn't about storing everything — it's about recalling the right thing at the right step. More storage doesn't fix a bad retrieval strategy.
Memory is one piece of a bigger loop
Memory feeds directly into the decisions an agent makes at each step of its loop, and it works alongside the tools an agent calls to gather new information in the first place. Understanding memory in isolation only goes so far — it matters most in the context of how an agent decides what to do next.
To see how memory fits into the bigger picture, the agent loop walks through the decide-act cycle memory feeds into, and what is rag covers the retrieval technique long-term memory is usually built on.
FAQ
What is memory in an AI agent?
Memory in an AI agent is the mechanism that carries relevant information forward — from earlier in the same task (short-term) or from past sessions entirely (long-term) — so the agent doesn't treat every step as if it's starting from scratch.
What's the difference between short-term and long-term agent memory?
Short-term memory covers the current task or session, usually held in the conversation history sent to the model. Long-term memory persists across separate sessions, typically stored in an external database and retrieved selectively when it's relevant to the current situation.
Do all AI agents have long-term memory?
No. Many simple agents only use short-term memory and start fresh each session. Long-term memory is an added capability, usually built with a vector database or similar storage, and it's most valuable for agents that need to recall user preferences or past outcomes across multiple sessions.
How is agent memory related to RAG?
Long-term agent memory is commonly built using the same retrieval technique as RAG: past information is stored as embeddings and retrieved by similarity when relevant. The difference is what's being retrieved — RAG typically retrieves from documents, while agent memory retrieves from the agent's own history of interactions.