Semantic Memory in AI Agents: Facts, Profiles, Graphs

Semantic memory in AI agents stores facts: who the user is, what they prefer, what’s true about the domain, and how entities relate. Unlike episodic memory, a semantic fact isn’t tied to the moment it was learned. The agent saves these facts outside the model and pulls the relevant ones into the prompt to ground and personalize its answers.

It’s the most common kind of agent memory in production, because most “the assistant remembers me” features are semantic memory. The hard part isn’t storing facts. It’s deciding what counts as a fact, and keeping facts correct when they change. The pillar guide, AI agent memory explained, shows where semantic memory sits among the other types.

What is semantic memory in AI agents?

Semantic memory in an AI agent is a long-term store of facts and concepts, detached from when they were learned: user attributes and preferences, domain knowledge, and relationships between entities. The agent writes facts into it from conversations or documents and retrieves them to ground its responses.

In psychology, semantic memory is general knowledge, distinct from episodic memory of specific events. The CoALA framework (Sumers et al., 2023) carries the term over to agents: semantic memory “stores an agent’s knowledge about the world and itself.”

Semantic memory is not semantic search

LangGraph’s memory guide makes this point directly. Semantic memory is a term from psychology for stored facts. Semantic search is a retrieval method that matches by meaning, usually with embeddings. You can implement semantic memory as a JSON profile you look up by user ID, with no vectors involved.

Where semantic memory comes from

CoALA describes two sources.

• Read-only corpora. Classic retrieval-augmented generation reads from human-written text such as Wikipedia, product docs or a catalog. CoALA calls this retrieving from “a semantic memory of unstructured text.” The agent never writes to it.
• Agent-written knowledge. Agents can also write new knowledge into semantic memory as a form of learning. A chat assistant extracts “the user is vegetarian” from a conversation. Reflexion (Shinn et al., 2023) reflects on a failed attempt and stores the lesson (“there is no dishwasher in kitchen”) for the next try.

The second source is what turns RAG into memory: the store changes with every interaction. The distinction is covered in more depth in RAG vs agent memory.

Facts often come from episodes. Generative Agents (Park et al., 2023) periodically reflects over raw events and writes conclusions like “I like to ski now,” which CoALA classifies as semantic memory. See episodic memory in AI agents for the event side.

Profile vs collection

LangGraph and LangMem name two ways to hold semantic memory. The choice affects extraction, updates and retrieval.

LangMem’s conceptual guide describes profiles as “a single document that represents the current state” and says collections “are what most people think of when they imagine agent long-term memory.” Many systems use both: a small profile always in the prompt, and a searchable collection for everything else.

How semantic facts are represented

Graphs help when questions span entities (“who on Bob’s team worked on the NLP project?”). Graphiti, for instance, stores facts as edges with validity windows so a fact can be superseded without being deleted. The trade-offs between stores are covered in knowledge graphs for AI memory.

Extracting semantic memories with LangMem

LangMem’s create_memory_manager prompts an LLM to extract facts into a schema, and with enable_deletes=True it can also remove facts that new information contradicts. This is adapted from LangMem’s semantic extraction guide:

 1from langmem import create_memory_manager
 2from pydantic import BaseModel
 3
 4class Triple(BaseModel):
 5    """Store all new facts, preferences, and relationships as triples."""
 6    subject: str
 7    predicate: str
 8    object: str
 9    context: str | None = None
10
11manager = create_memory_manager(
12    "anthropic:claude-sonnet-4-5",
13    schemas=[Triple],
14    instructions="Extract user preferences and any other useful information",
15    enable_inserts=True,
16    enable_deletes=True,
17)
18
19first = [{"role": "user", "content": "Alice manages the ML team and mentors Bob, who is also on the team."}]
20memories = manager.invoke({"messages": first})
21# -> Triples like (Alice, manages, ML_team), (Alice, mentors, Bob), (Bob, is_member_of, ML_team)
22
23second = [{"role": "user", "content": "Bob now leads the ML team and the NLP project."}]
24update = manager.invoke({"messages": second, "existing": memories})
25# -> RemoveDoc for (Alice, manages, ML_team), plus new triples (Bob, leads, ML_team), (Bob, leads, NLP_project)

This functional API doesn’t touch a database. You decide what a RemoveDoc means: a hard delete, a soft delete, or a lower ranking. To have LangMem write to a LangGraph store directly, use create_memory_store_manager with a namespace.

The guide makes a point worth copying: add a context field. A memory like {"content": "User said yes"} is useless later. (user, response, yes, "when asked about attending team meeting") is not.

Keeping semantic memory current

Facts change. “I live in NYC” in January becomes “I just moved to SF” in June. If both sit in the store, retrieval may return either. Systems handle this in a few ways:

1. Overwrite. Profiles replace the old value. Simple, but history is gone.
2. Delete on contradiction. LangMem’s manager emits RemoveDoc for facts the new message contradicts.
3. Invalidate with time. Graphiti marks the old edge invalid from the moment the new fact became true, so “where did Alice live in March?” still works.
4. Keep both, rank by recency. Cheap, but pushes conflict resolution onto the model at read time.
5. Consolidate into beliefs. Some systems merge related facts into a summary that is refined as evidence comes in. Hindsight calls these observations: each one keeps quotes from its source facts and a proof count, and gets refined rather than overwritten when a new fact arrives.

Test this case with real data before you pick a system. LongMemEval (Wu et al., 2024) has a dedicated “knowledge update” category for it, because it’s where many setups fail quietly.

How memory systems score on facts that change

Two LongMemEval question types test semantic memory directly. Knowledge-update (78 questions) asks for a fact the user later changed. Single-session-preference (30 questions) asks the assistant to use a preference the user stated once. Table 3 of the Hindsight paper (Latimer et al., arXiv 2512.12818, Dec 2025) breaks results down by type, on the S setting (about 115,000 tokens over roughly 50 sessions):

What the table shows:

• Putting the whole history in the prompt is worst at preferences. Full context got 20% on both models: the stated preference is in there, but the model doesn’t apply it. Every memory system did at least 2.8x better, which is the case for storing preferences as separate facts.
• Updates are where systems differ most at the top. Hindsight with Gemini-3 answered 94.9% of knowledge-update questions and Supermemory with Gemini-3 89.7%. On preferences, Hindsight’s best was 86.7% with the open OSS-120B model.
• The answer model matters as much as the memory. Hindsight goes from 84.6 to 94.9 on updates just by changing the model that reads the memories.

Read it with three limits. The paper is by Hindsight’s authors at Vectorize, so this is a vendor-run comparison, not an independent one. The question counts are small: on preferences one question is 3.3 points. And LongMemEval’s judge counts an answer as correct if it gives the new value, even when it also repeats the old one, so the test doesn’t punish a system for keeping stale facts around. Hindsight’s benchmarks site now reports 94.6 overall on LongMemEval S, newer than this table. The LLM memory comparison collects other published results.

Retrieving semantic memory

How facts reach the prompt matters as much as how they’re stored:

• Always-on profile. Load a small profile into the system prompt on every turn. No search, no misses, fixed token cost.
• Query-time search. Search the collection with the user’s message. Combine embeddings with keyword search, since names, product codes and IDs often fail pure vector search.
• Entity lookup. Detect entities in the message and fetch their facts or graph neighborhood.
• Agent-driven tools. Give the agent a search tool (LangMem’s create_search_memory_tool) and let it decide when to look things up.
• Several searches at once. Hindsight’s recall runs semantic, BM25 keyword, graph and time-range search in parallel, fuses the results, reranks them with a cross-encoder and trims them to a max_tokens budget. It’s strong on long histories (73.9 on BEAM 1M, vendor-reported) but returns more context than Mem0 by default, so set that budget on purpose.

Keep the memory block small. A few precise facts beat twenty loose matches, and every irrelevant fact is a chance for the model to get distracted.