How AI Agent Memory Works
How AI agents remember: the context window, saved summaries, searchable long-term memory and instruction files, plus the risks and how to stay in control.

If you used early AI chatbots, you know the "goldfish effect". You'd spend twenty minutes explaining a problem, start a new chat, and the AI had forgotten everything. It was like hiring a new intern every morning.
Many of today's AI agents can carry information forward instead: your preferences, the decisions made on a project, what went wrong last time. But the model inside an agent doesn't change when you talk to it. Its weights are fixed. So where does the memory live?
The short answer: outside the model. Agent memory is information the surrounding software saves, then feeds back into the model's input at the right moment. Understanding that one idea explains both what memory can do and where it goes wrong.
An AI model is like a brilliant colleague with no long-term memory. Agent memory is the notebook the colleague is handed at the start of every meeting: what's in the notebook, and how well it's organised, decides what they "remember".
The four kinds of agent memory
Researchers and builders borrow terms from human psychology. Different products mix these in different ways.
Short-term memory: the context window
The context window is everything the model can see in one go: your messages, its replies, instructions and the output of any tools it used. It's fast and precise, but limited in size and measured in tokens.
When it fills up, something has to give. Agents typically summarise older parts of the conversation (often called compaction), drop tool output they no longer need, or save large results to a file and keep only a pointer. Overfilled context makes answers slower, costlier and more error-prone.
Episodic memory: what happened before
Episodic memory is a record of past events: tasks attempted, what failed and what fixed it.
Say an agent tried to book travel last month and the payment failed because a card had expired; it asked you for a new one. If that episode was saved, next time it can check the card before starting. In practice this is often a log or a set of short notes the agent writes at the end of a task, which later sessions can search.
Semantic memory: facts and preferences
Semantic memory stores facts that stay true: "This user writes in British English", "The marketing budget never exceeds $5,000", "The API is on version 3".
These facts are usually saved as text and made searchable with embeddings in a vector database, so the agent can find the relevant ones with semantic search when you ask something related. Some systems also use a knowledge graph to record how people, projects and decisions connect. This is the same machinery as RAG, pointed at the agent's own notes.
Procedural memory: how to do things
Procedural memory is know-how: the steps, rules and conventions for recurring work. For agents, this is often the plainest memory of all, a file of instructions the agent reads at the start of every session. Coding agents commonly use a project file such as
AGENTS.md("run the tests with this command", "never edit the migrations folder"), and many tools let you save reusable instructions or "skills".Kind Holds Typical storage Lasts Short-term The current conversation and tool output The context window One session Episodic What happened on past tasks Logs, progress notes, summaries Across sessions Semantic Facts and preferences Notes indexed in a vector database or knowledge graph Until updated or deleted Procedural How to do recurring work Instruction files, saved prompts, skills Until edited
How memory flows through an agent
Most memory systems follow the same loop:
- Write: during or after a task, the agent (or the software around it) decides what's worth saving, such as a preference you mentioned or the fix for a bug.
- Store: the note is saved with details like when it was written and where it came from.
- Retrieve: at the start of a new task, the system searches memory for notes related to the request.
- Use: the best matches are added to the model's input alongside your message.
- Update or forget: when a fact changes ("we moved from React to Vue"), the old note should be replaced, not kept alongside the new one.
All of this is part of the agent's harness, the software around the model. That's why two agents using the same model can remember very differently. (More in What is harness engineering?)
Curious how agents connect to databases?
Read our guide to MCP to understand how AI models pull information from external storage systems.
Where agent memory goes wrong
- Stale facts. If an old preference isn't removed, the agent may act on both the old and the new one, or on the wrong one.
- Wrong facts. A misunderstanding saved once can be repeated in every later session. Memory makes mistakes persistent as well as successes.
- Over-remembering. Pulling too many notes into each request crowds the context window and can drag in irrelevant details.
- Memory poisoning. If an agent saves text from web pages or emails, a hidden instruction can end up in its memory and resurface later, a long-lived form of prompt injection. Good systems save summaries in their own words and treat outside content as untrusted.
- Privacy. Memory may hold personal or confidential information. It needs the same care as any other stored data.
Staying in control of what an agent remembers
Whether you're using an assistant or building one, look for:
- Visibility: can you see what's been saved?
- Editing and deletion: can you correct or remove a memory, or turn memory off?
- Scope: is memory kept per project or per workspace, so one client's details don't leak into another's work?
- Sources and dates: does each memory record where and when it came from?
- Plain instruction files for anything that must always apply. They're easier to review than automatic memory.
Frequently asked questions
Does an AI agent learn from me over time?
Not in the sense of the model changing. The agent saves notes and feeds them back later. The model itself stays the same unless its developer retrains it.
Where is an agent's memory stored?
Outside the model: in files, databases or the product's servers. Short-term memory lives only in the current context window.
What's the difference between agent memory and RAG?
They use the same retrieval techniques. RAG usually searches documents you provide; agent memory searches notes the agent wrote about past work and preferences.
Can I make an agent forget something?
Usually yes, if the product exposes its memory: delete the note or clear memory. For instruction files, edit or remove the line.
Is a bigger context window the same as memory?
No. A larger window lets an agent see more at once, but it still resets between sessions. Memory is what carries information across them.
Summary
- An agent's model doesn't change; memory lives in the software around it.
- Four kinds work together: short-term (context window), episodic (past events), semantic (facts) and procedural (instructions).
- Memory is written, stored, retrieved, used and updated in a loop.
- It can go wrong through stale or wrong facts, overload, poisoning and privacy gaps.
- Prefer tools that let you see, edit, scope and delete what they remember.

Stephen Gbolagade
Software Engineer and Technical Writer at Ploba
Writes Ploba's guides to AI tools, agents and protocols, explaining how they work in plain language.
Keep learning
Plain-language explanations, technical guides and practical comparisons.


