How to Add Memory to a Chatbot (with Python)

To add memory to a chatbot, store each conversation’s messages and resend them with every request, then trim or summarize old turns so the history fits the model’s window. For memory across conversations, also save key facts about the user to a database and add the relevant ones to the prompt when a new chat starts.

What is chatbot memory?

Chatbot memory is everything an app sends to the model so it can respond as if it remembers: the recent message history of the current chat (short-term memory) and stored facts from past chats (long-term memory). The model itself keeps nothing. Every API request starts blank unless your code puts the past back in.

That’s why a raw API call “forgets” your name one message later. Chat apps hide this by resending the history. You have three places to keep it:

• In your app. You store messages and send them each time. Full control, works with any model.
• With the provider. OpenAI’s Responses API can chain turns with previous_response_id or keep a durable Conversation object. The OpenAI conversation state guide notes that “all previous input tokens for responses in the chain are billed as input tokens,” so this saves code, not money.
• In a framework. LangGraph checkpointers save the message state per thread.

Long-term memory is a different job. It needs a store keyed by user, not by conversation. The broader picture is in AI agent memory explained.

Chatbot memory strategies compared

Resending the full history works for short chats. Long chats need a strategy, because cost grows with every turn and the history eventually overflows the context window.

Longer prompts aren’t free in quality either. The Lost in the Middle paper (Liu et al., 2023) found that model performance “significantly degrades when models must access relevant information in the middle of long contexts.” A short summary plus recent turns often beats pasting the whole transcript.

Most production chatbots combine the last three: a token budget for recent turns, a summary of older ones, and a small block of long-term facts.

How to add memory to a chatbot in 6 steps

1. Give every conversation an ID. A session_id or thread_id. Memory without an ID mixes up users.
2. Persist messages. Write each user and assistant message to a database as it happens. In-process lists vanish on restart.
3. Load history per request. Read the session’s messages and send them after the system prompt.
4. Cap the history. Keep recent turns within a fixed count or token budget.
5. Summarize what you drop. Fold old turns into a running summary and put it in the system prompt.
6. Add long-term facts if you need them. Save stable facts per user and retrieve them at the start of each chat.

Example: persistent chat memory with SQLite and a rolling summary

This is the whole pattern in plain Python with the OpenAI SDK and SQLite: history survives restarts, the last six messages go verbatim, and older turns get folded into a summary.

Tested with openai 3.26:

 1import sqlite3
 2from openai import OpenAI
 3
 4client = OpenAI()
 5MODEL = "gpt-5-mini"
 6KEEP_TURNS = 6  # recent messages sent verbatim
 7
 8db = sqlite3.connect("chat.db")
 9db.executescript("""
10CREATE TABLE IF NOT EXISTS messages (session_id TEXT, role TEXT, content TEXT, id INTEGER PRIMARY KEY);
11CREATE TABLE IF NOT EXISTS summaries (session_id TEXT PRIMARY KEY, summary TEXT);
12""")
13
14def load(session_id: str) -> tuple[str, list[dict]]:
15    row = db.execute("SELECT summary FROM summaries WHERE session_id = ?", (session_id,)).fetchone()
16    rows = db.execute("SELECT role, content FROM messages WHERE session_id = ? ORDER BY id", (session_id,))
17    return (row[0] if row else ""), [{"role": r, "content": c} for r, c in rows]
18
19def save(session_id: str, role: str, content: str) -> None:
20    db.execute("INSERT INTO messages (session_id, role, content) VALUES (?, ?, ?)", (session_id, role, content))
21    db.commit()
22
23def compact(session_id: str, summary: str, history: list[dict]) -> None:
24    """Fold everything except the last KEEP_TURNS messages into a running summary."""
25    old = history[:-KEEP_TURNS]
26    if not old:
27        return
28    transcript = "\n".join(f"{m['role']}: {m['content']}" for m in old)
29    resp = client.chat.completions.create(model=MODEL, messages=[{
30        "role": "user",
31        "content": f"Update this summary of a chat. Keep names, facts, decisions and open questions.\n\n"
32                   f"Current summary:\n{summary or '(none)'}\n\nNew messages:\n{transcript}",
33    }])
34    db.execute("INSERT OR REPLACE INTO summaries VALUES (?, ?)", (session_id, resp.choices[0].message.content))
35    db.execute("DELETE FROM messages WHERE session_id = ? AND id NOT IN "
36               "(SELECT id FROM messages WHERE session_id = ? ORDER BY id DESC LIMIT ?)",
37               (session_id, session_id, KEEP_TURNS))
38    db.commit()
39
40def chat(session_id: str, user_message: str) -> str:
41    save(session_id, "user", user_message)
42    summary, history = load(session_id)
43    system = "You are a helpful assistant."
44    if summary:
45        system += f"\n\nSummary of the earlier conversation:\n{summary}"
46    resp = client.chat.completions.create(model=MODEL, messages=[{"role": "system", "content": system}, *history])
47    answer = resp.choices[0].message.content
48    save(session_id, "assistant", answer)
49    if len(history) + 1 > 2 * KEEP_TURNS:
50        compact(session_id, summary, history + [{"role": "assistant", "content": answer}])
51    return answer

In our test, we told the bot “I’m Ana, planning a 5-day trip to Japan in April,” then sent six more messages. The first turns were compacted into the summary, and a later “What’s my name and where am I going?” was still answered correctly: “Your name is Ana and you’re going to Japan.”

Two choices to tune: KEEP_TURNS (higher means more exact recall, more tokens) and the summary prompt (tell it what your domain must never lose, such as order numbers or dates).

Example: the same with LangChain and LangGraph

If you use LangChain, the old memory classes are gone from the main package. ConversationBufferMemory now lives in langchain-classic and warns that it “was deprecated in LangChain 0.3.1 and will be removed in 2.0.0. Use langchain.agents.create_agent instead.” The replacement is an agent with a checkpointer (saves each thread’s messages) and SummarizationMiddleware (compacts long threads).

Tested with langchain 1.4 and langgraph-checkpoint-sqlite:

 1import sqlite3
 2from langchain.agents import create_agent
 3from langchain.agents.middleware import SummarizationMiddleware
 4from langgraph.checkpoint.sqlite import SqliteSaver
 5
 6checkpointer = SqliteSaver(sqlite3.connect("checkpoints.db", check_same_thread=False))
 7
 8agent = create_agent(
 9    model="openai:gpt-5-mini",
10    tools=[],
11    checkpointer=checkpointer,
12    middleware=[SummarizationMiddleware(model="openai:gpt-5-mini", trigger=("tokens", 4000), keep=("messages", 20))],
13)
14
15config = {"configurable": {"thread_id": "ana-session-1"}}
16agent.invoke({"messages": [{"role": "user", "content": "Hi, I'm Ana."}]}, config)
17result = agent.invoke({"messages": [{"role": "user", "content": "What's my name?"}]}, config)
18print(result["messages"][-1].content)  # "Your name is Ana. ..."

The LangChain short-term memory docs recommend a database-backed checkpointer such as PostgresSaver in production. A fuller walkthrough is in building a chatbot with memory in LangGraph.

Adding long-term memory across conversations

History and summaries are per conversation. When the user opens a new chat, they’re gone. To remember a user across chats, add a second store keyed by user ID:

• After each conversation (or turn), extract stable facts: name, preferences, ongoing projects. An LLM call with “list lasting facts about the user” works; memory libraries like Mem0 do this for you.
• At the start of each chat, retrieve the facts that match the first message and add them to the system prompt.
• When facts change, update or replace them so “lives in NYC” doesn’t survive the move to SF.

Tested code for both the do-it-yourself and library versions is in how to give AI agents memory. If you’d rather not build it, memory libraries handle extraction and updates for you:

• Mem0: the lightest option. One LLM pass per write, under 7K tokens per retrieval by its own measurement.
• Zep: a managed service that tracks when facts change, so “lives in NYC” gets a validity window instead of lingering.
• LangMem: memory tools inside LangGraph, if your chatbot already runs there.
• Hindsight: an MIT-licensed server built for long histories. It scores 73.9 on BEAM at 1M tokens and 64.1 at 10M, against 64.1 and 48.6 for Mem0 (both vendor-reported, from Mem0’s comparison page). It merges related facts into observations that get refined as new ones arrive. The catch: writes and consolidation call an LLM, and it returns more context per retrieval than Mem0, so set a token budget. For a simple FAQ bot it’s more than you need.

How to clean up chatbot memory

Memory you can’t delete becomes a privacy problem and a quality problem. Plan cleanup in three layers:

• Session history. Delete the session’s rows (DELETE FROM messages WHERE session_id = ?) and its summary. With LangGraph checkpointers, delete the thread.
• Long-term facts. Delete by user ID. Mem0 has delete_all(user_id=...); most memory APIs have an equivalent.
• Bad memories. When a stored fact is wrong, fix or delete that one item rather than letting it be retrieved forever. Keep the source message ID with each fact so you can trace where it came from.

If you’re a user rather than a builder: in ChatGPT you can view and delete saved memories, or turn memory off. Steps are in how to find ChatGPT memory.