Databricks wants agents to remember what they learned yesterday. The hard problem is what they should forget

Give an agent everything it has ever seen, and it does not get much better. On the open benchmarks in Databricks’ new memory research, an agent fed its full conversation history scored 87.8%. The same agent with a curated memory scored 88.1%, while reading about 8,400 input tokens per question instead of 103,000. On the benchmarks with the longest histories, the full context did worse: the extra material confused it.

That result is the most useful thing in Databricks’ October 20 launch of Agent Memory, now in beta for agents built on its platform. Remembering is easy. The design work is in deciding what to keep, what to rewrite and what to drop.

What Databricks shipped

Agent Memory stores three things in Lakebase, Databricks’ serverless Postgres, provisioned in the customer’s workspace and operated by Databricks:

• Managed sessions: the ordered history of messages, tool calls and results.
• Managed memory: long-term facts about a user’s profile and preferences, domain knowledge, and workflows that worked.
• Memory pipelines: background jobs, on demand or on a schedule, that distill sessions into memory. They extract durable facts, remove duplicates and replace facts that newer conversations contradict.

Personal memories are scoped to each user. Developers can read, export and delete what an agent keeps about each user, and can instruct the pipelines on what to retain.

The research behind it, MemCurrent, separates episodic memory (raw, append-only session traces) from derived memory (distilled learnings kept as files in a hierarchical knowledge base, with an index loaded at the start of each session). An offline consolidation pipeline, which Databricks calls “dreaming”, merges duplicates and resolves conflicts.

The numbers, with their scope

On an internal benchmark built from real conversations with Databricks’ Genie data agent (114 test cases, 29 users, 2,233 historical sessions), MemCurrent reached a 68.4% pass rate. With no memory the agent passed 22.3%; with Mem0, a popular memory backend, 43.9%; with the correct memories injected by hand, the ceiling was 78.6%. Against no memory, MemCurrent tripled the pass rate and ran 1.35 times faster at 1.44 times lower cost.

Two cautions. The benchmark comes from one workspace and one agent, and it is Databricks’ own. And the most interesting ablation is easy to miss: derived memory alone scored 66.1%, episodic memory alone 35.8%. Almost all the value came from the curated layer, not from keeping the raw history.

Three kinds of agent state, three different owners

Episodic history

• Raw session traces
• Tool calls and results
• Append-only

Evidence of what happened

Derived memory

• Profile and preferences
• Domain knowledge
• Workflows that worked

Rewritten by a pipeline; can be wrong

System of record

• Customer master
• Approved procedures
• Transactions

Never replaced by memory

The forgetting problem

Persistent memory turns an agent into a data store, as we argued when we described agent memory as enterprise data storage. Agent Memory makes that concrete, and raises four questions the announcement does not fully answer.

Staleness. A consolidation pipeline that replaces contradicted facts handles the case where a user says something new. It does not handle the case where the world changes and nobody tells the agent: a reorganization, a new pricing rule, a supplier dropped. Derived memory needs an expiry or a revalidation rule, not only a conflict rule.

Consolidation errors. A pipeline that rewrites memory can also get it wrong, merging two distinct facts or keeping an exception as a rule. Databricks describes what the pipeline does, not how its mistakes are detected. Teams should sample consolidated entries the way they sample any automated output.

Retention of the raw layer. Episodic memory is append-only, and neither post describes a retention schedule for it. Under a privacy request, deleting a distilled preference is not enough if the transcript that produced it is still there. Wherever people have a legal right to deletion, from California to the EU, that is a legal question, not a tuning question.

Scope. User-scoped memory is clear. Shared domain knowledge and workflows are not: who approves that a workflow one user taught the agent becomes the way it works for everyone? Anthropic’s sales team, in its six-agent setup, shared memory deliberately and kept credentials separate. That separation should be a design decision, not a default.

Memory also makes an agent harder to move. What it has learned now lives in one vendor’s store, which is why accumulated state is the new switching cost. The export function matters; test it before the memory is worth anything.