For the last two years the default answer to "how does my agent remember things" was: add a vector database. That answer is now under real pressure. New benchmarks, a native memory primitive from Anthropic, and a fast-moving crop of memory frameworks are forcing teams to separate what memory actually is from what retrieval-augmented generation has always done, and the early evidence favors something much dumber than embeddings and cosine similarity.
Memory and RAG are not the same problem
Most teams still conflate the two. RAG solves a retrieval problem over a fixed or slowly-changing corpus: documents, tickets, code, policy text. You embed it, you search it, you stuff the top-k results into a prompt. Agent memory solves a different problem: what should the agent itself remember about a conversation, a user, or a multi-day task, and how does it decide what to forget. That includes episodic facts ("the user prefers Slack over email"), procedural learnings ("this API returns 429s if you batch over 50"), and working state for a task that spans more context than any window can hold. Treating memory as just another RAG corpus is why so many agent memory implementations feel bolted on. They retrieve well but never actually update, consolidate, or prune anything, so the memory store just grows into a second, worse vector index.
The filesystem result that should worry the vector-DB defaults
Letta published a benchmark result this year that deserves more attention than it got: a plain filesystem-based memory approach, where conversational history is simply written to files and read back, scored 74.0% on the LoCoMo long-conversation memory benchmark, outperforming specialized memory tool libraries built specifically for this job. There is no fancy embedding model, no re-ranker, no graph store. Just structured files, written and read with normal tool calls.
This matters because it directly rebuts the reflexive "just add a vector DB" answer that a lot of teams give to memory problems without measuring anything. Vector similarity search is good at finding semantically related content across a large corpus. It is not obviously the right tool for recalling a small number of durable facts about a specific user or task, where exact retrieval of a known key beats approximate retrieval of a probable match. A file with a clear schema, read deterministically, sidesteps an entire class of retrieval failures: near-miss embeddings, chunking artifacts, and reranking that surfaces the wrong but similar-sounding memory.
A memory layer that locks you to one framework is a memory layer developers won't adopt at scale.
What Anthropic's memory tool actually does
Anthropic shipped a native memory tool and context editing capability on the Claude API this fall, and it's worth being precise about what problem it solves. It is not retrieval over documents. It lets an agent write information to files that persist across context resets, so a long-running task can clear its working context without losing the facts and state it accumulated. When the context window fills up, the agent edits it down, having already saved what it needs to a file it can read back later.
This is architecturally close to what Letta's filesystem result found by accident: memory as agent-managed files the model reads and writes with normal tool calls, not memory as a separate retrieval pipeline sitting alongside the model. The practical implication for teams building on Claude is that a fair amount of what used to require a dedicated memory service can now be handled with file writes and a clear naming convention, at least for the working-memory and session-continuity use case. It does not replace retrieval over a large knowledge base. It replaces the pattern of cramming everything back into the prompt every time the agent needs to recall something from three tool calls ago.
The benchmarks are quietly damning
Separate from the filesystem-versus-vector-DB argument, a set of new academic benchmarks is exposing how bad current models are at long-horizon recall generally. LongMemEval found commercial chat assistants and long-context LLMs show roughly a 30% accuracy drop compared to being handed the correct context directly, meaning models perform substantially worse when they have to find the right memory themselves versus when someone hands it to them on a plate. MemoryAgentBench tells a similar story from a different angle: agents that look fine in short interactions degrade sharply once the task requires consolidating information across many turns or sessions.
This is a distinct failure mode from in-context degradation over a single long conversation. It's not that the model gets confused by too much text in front of it right now. It's that the model, given a memory system and asked to retrieve the right fact from it, picks wrong at a meaningfully higher rate than if you'd just told it the fact. That's a sobering number for anyone assuming a memory layer makes an agent reliably smarter over time. In practice it often just adds a new place for the agent to be wrong.
A crowded field with no settled default
Teams shipping agents right now are choosing a memory stack with no dominant standard to lean on. The field includes:
- Mem0, with over 51,000 GitHub stars and $24M raised as of late 2025, positioned as a framework-agnostic memory layer
- Letta, the team behind the filesystem benchmark, offering both managed memory and the underlying research
- Zep, focused on temporal knowledge graphs for agent memory
- Cognee, targeting structured knowledge extraction as a memory substrate
- LangChain's LangMem SDK, now a first-class long-term memory primitive in LangGraph, separate from short-term conversation state
- Cloudflare's own agent memory offering, built into their edge agent runtime
- Anthropic's native memory tool and context editing, built directly into the Claude API
Mem0's own "State of AI Agent Memory" report makes the fragmentation explicit, noting 13 separate agent framework integrations documented for their product alone. That number is a reasonable proxy for how unsettled the whole space is. When a memory vendor has to maintain 13 integration paths just to be usable across the frameworks people actually build with, there is no standard yet, and anyone picking a memory layer today is picking without the benefit of a shaken-out winner.
Choosing a memory stack without locking yourself in
Given the fragmentation, the practical risk isn't picking a bad memory tool, it's picking one that couples your agent logic tightly to a specific vendor's schema and API before you know what your actual recall patterns look like. A few principles hold up regardless of which library wins the next eighteen months:
- Keep the memory read/write interface behind your own abstraction, not the vendor's SDK calls sprinkled through agent code
- Separate short-term working state (what happened in this session) from long-term memory (what should persist across sessions) explicitly, even if you store them in the same place for now
- Instrument recall accuracy before you scale the memory store, because most teams never measure whether the agent retrieves the right memory, they just assume it does
- Prefer memory formats you can inspect and edit by hand (structured files, plain records) over opaque embeddings you can only query
Start simple, add retrieval infrastructure only when you've proven you need it
The Letta filesystem result and Anthropic's file-based memory tool point in the same direction, and it's a useful, slightly contrarian starting point for teams under pressure to stand up a full memory database on day one. Most agents don't need semantic search over thousands of accumulated memories. They need a small number of durable, structured facts, reliably written and reliably read back. A JSON file per user or per task, updated by the agent through normal tool calls, gets most teams further than a vector database with a reranker bolted on top, and it's dramatically easier to debug when recall fails, because you can just open the file and look.
The moment to reach for a dedicated memory layer is when you have evidence the simple approach breaks: memory volume that no longer fits in a manageable file, retrieval patterns that genuinely require semantic similarity rather than exact lookup, or multiple agents needing shared, concurrently-updated memory. Building that infrastructure before you have that evidence just adds a second system that can silently return the wrong context, on top of a model that, per the current benchmarks, is already meaningfully worse at self-directed recall than we'd like to believe.