Agent memory is everything an AI agent keeps beyond a single model call so it can act coherently: the messages and tool results from the current task (short-term memory), facts and preferences it should remember across sessions (long-term memory), and the state that records where a multi-step workflow currently stands. Language models are stateless—each API call starts from zero—so memory and state are something the engineer designs, stores, and feeds back into the prompt.

Why it matters: as soon as an agent runs more than one step, calls tools, or talks to the same user twice, memory decides whether it feels reliable or forgetful. It is also where many production bugs hide: bloated context that raises cost and latency, stale facts that contradict new ones, and data from one user leaking into another user’s session.

Memory vs state: two related but different ideas

Developers often use the words interchangeably, but separating them makes design decisions easier:

• State is the structured snapshot of a running workflow: the current step, intermediate results, retries, flags such as “awaiting human approval.” It is usually typed (a dict or schema) and saved after every step so a run can pause, resume, or recover from a crash.

• Memory is the information the agent can recall to make better decisions: conversation history, summaries, user preferences, past outcomes, and learned instructions.

Frameworks reflect this split. In LangGraph, short-term memory is part of a graph’s state and is persisted per conversation thread by a checkpointer, while long-term memory lives in a separate store that can be shared across threads (LangGraph: Memory concepts).

The main types of agent memory

The research framework CoALA (Cognitive Architectures for Language Agents) borrows from cognitive science and describes a working memory plus three long-term memory types (Sumers et al., CoALA). It is a useful map for engineers:

Short-term memory techniques

• Full buffer: send every message back each turn. Simple, but cost and latency grow with every step.

• Sliding window: keep only the last N messages or tokens.

• Summarization: periodically compress older turns into a running summary.

• Selective trimming: drop bulky tool outputs once their result has been extracted into state.

More context is not automatically better. The “Lost in the Middle” study found that models use information at the beginning and end of long inputs more reliably than information buried in the middle (Liu et al., Lost in the Middle). Curating what goes into the window usually beats stuffing it.

Long-term memory techniques

• Profile memory: a small, structured record per user (name, role, preferences) that the agent reads at the start of a session and updates when it learns something new.

• Retrieved memory: past notes or conversations embedded and stored in a vector index, then retrieved by similarity. If you are new to this, start with what vector databases are and why AI agents need them.

• Tiered memory: systems such as MemGPT treat the context window like RAM and external storage like disk, letting the model decide what to page in and out (Packer et al., MemGPT).

How to add memory and state to an agent, step by step

1. Define the state schema first. List the fields each step reads and writes (for example messages, plan, tool_results, status). Typed state makes bugs visible.

2. Persist state per thread. Use a checkpointer backed by a database or Redis, keyed by a thread or session ID, so a run can resume after a timeout or a human-in-the-loop pause. Redis documents several patterns for AI session and memory storage (Redis for AI).

3. Decide what deserves long-term memory. Not every message is worth remembering. Write explicit rules: store stable preferences and confirmed facts; skip small talk and one-off details.

4. Choose when to write. Writing “in the hot path” (during the conversation) is immediate but adds latency; writing in a background job after the session is cheaper and lets you deduplicate.

5. Scope memory by user and tenant. Namespace every record so one customer’s data can never be retrieved for another. This is a security requirement, not an optimization.

6. Retrieve narrowly. Pull only the few memories relevant to the current request and label them clearly in the prompt so the model knows they are recalled context, not new instructions.

7. Expire and correct. Add timestamps, let newer facts overwrite older ones, and give users a way to see or delete what is stored.

Memory interacts closely with tools: an agent that calls APIs needs to remember which calls already succeeded. See what function calling is and why agents depend on tool use for the other half of the picture.

Common failure modes (and how to test for them)

• Context bloat: token counts creep up every turn. Track tokens per step and set budgets.

• Stale or contradictory memory: the agent recalls an old address after the user changed it. Test with scripted conversations that update a fact and check which version wins.

• Cross-user leakage: run tests with two users and assert that retrieval never crosses namespaces.

• Memory injection: a malicious message gets saved and later treated as an instruction. Store memories as data, sanitize them, and apply guardrails on what can be written.

• Lost progress: a crash mid-workflow restarts from scratch. Kill the process during a test run and confirm it resumes from the last checkpoint.

Traces are the best debugging tool here: log what was retrieved, what was written, and what the model saw at each step, then review failing runs.

What to look for in a course that teaches memory and state

Memory is hard to learn from slides because the bugs only appear when you build and break real agents. When comparing instructor-led AI agents courses, look for:

• A dedicated module on agentic workflows, memory, and state—not just a mention inside a prompt engineering lesson.

• Hands-on work with a real framework (for example LangGraph) and a persistence layer such as Redis or a database.

• Evaluation and observability practice, so you can measure whether memory actually improves results.

• A capstone reviewed by an instructor, where you have to explain why your agent failed and how you fixed it.

Coderhouse’s AI Engineering course is built for developers already comfortable with Python: eight weeks with one two-hour live class per week (plan for 6–8 hours of total work weekly), including a week on agentic workflows, tools, memory, and state, and a capstone where you build, break, evaluate, and improve a system. It lists a $595 launch price, and you can try two classes free before committing.

FAQ

Do LLMs remember previous conversations on their own?

No. A model only sees what is in the current request. Any “memory” comes from the application storing information and sending it back.

Is RAG the same as long-term memory?

They use similar machinery—embeddings and retrieval—but RAG usually reads from a fixed knowledge base, while agent memory is written by the agent itself as it learns about users and tasks.

Where should agent state be stored in production?

In a durable store keyed by thread or session: a relational database, Redis, or the checkpointer backend your framework supports. In-memory storage is fine only for prototypes.

Do I need memory for every agent?

No. Single-shot tasks may only need state for the current run. Add long-term memory when users return and personalization or continuity clearly improves outcomes.

Want to build agents that remember the right things and recover when something breaks? Explore these Coderhouse courses:

• AI Engineering Course — build RAG systems and tool-using agents with memory, state, evaluation, and deployment.

• Data Analytics Course — learn the SQL, Python, and measurement skills to analyze how your systems and users behave.