Agent memory is enterprise data storage, even when nobody calls it a database

Most enterprise agents today are forgetful by design. Each session starts empty, retrieves what it needs and ends. Product teams increasingly want the opposite: an agent that remembers what it learned last week, connects it with what it read today and gets better over time. That is a real capability. It is also a new data store, with all the obligations that come with one.

What Google Cloud’s sample does

Google Cloud’s generative AI examples repository includes an always-on memory agent, built with the Agent Development Kit and Gemini 3.1 Flash-Lite. It works in three phases:

• Ingest. Text, images, audio, video and PDFs (27 file types in five categories) are turned into structured memories with summaries, entities and importance scores. Content arrives through a watched inbox folder, a dashboard upload or an HTTP endpoint.
• Consolidate. Every 30 minutes by default, the agent reviews new memories, finds connections between them, generates cross-cutting insights and compresses related information.
• Query. Questions are answered by reading the memories and consolidation insights, with citations to the sources.

Memories live in SQLite. The design runs continuously in the background, which is why it uses a fast, inexpensive model.

To be precise about status: this is sample code in a public repository, not a managed Google Cloud product. It is a design reference worth studying, not something to deploy as is.

Why memory changes the risk model

Retrieval asks “what can the agent read right now?” Memory adds four harder questions:

What may it keep? Ingestion copies content out of its source system. Whatever classification and access rules the source had do not travel automatically.

For how long? A memory store without a retention rule keeps everything forever, including what the source system later deleted.

Who can recall it? In the sample, a query reads across all memories. That is fine for a single user’s assistant. In a shared enterprise agent, recall has to be permission-aware, or one person’s confidential input becomes another person’s answer.

Can it be corrected or erased? Consolidation creates derived data: insights and compressed summaries that combine several sources. Deleting a source document does not delete the insight that was built from it unless the system tracks provenance.

1. Ingest with source and classification
2. Store with owner and retention
3. Consolidate, keeping provenance
4. Recall filtered by caller's permissions
5. Correct or delete, including derived insights

• Write: Ingest with source and classification · Store with owner and retention
• Derive: Consolidate, keeping provenance
• Read and erase: Recall filtered by caller's permissions · Correct or delete, including derived insights

Labels travel with the dataEvery insight knows its sources

The design rules we would apply

Carry the source’s labels. Every memory should record where it came from, its sensitivity label and who was allowed to see the original. That is what makes permission-aware recall possible.

Keep provenance through consolidation. An insight should list the memories it was built from. Without that link, you cannot honor a deletion request or explain an answer.

Scope memory per tenant, team or user. Decide the boundary before the first byte is stored. Merging memories across contexts is a product decision with privacy consequences, not a performance optimization.

Set retention and test deletion. Pick a retention period per memory type and run a deletion drill that proves derived insights go too.

Put memory on the custody map. Where the store lives, who operates it and under which jurisdiction is the custody question applied to a new copy of your data. If you already run managed retrieval, the same reasoning as in our analysis of build-or-buy for retrieval applies.

For U.S. companies, two existing obligations reach agent memory directly. Under the California Consumer Privacy Act, a verified deletion request covers personal information wherever the business holds it, which includes an agent’s memory and anything derived from it. And memory is electronically stored information: when litigation is anticipated, a legal hold may require preserving it, which conflicts with aggressive automatic consolidation unless the system can pause it.

What to do now

1. Find the memory you already have: chat histories, agent session stores, vector indexes and “notes” features in tools your teams use.
2. Assign each an owner, a classification and a retention period.
3. Require provenance for any consolidation or summarization feature you build or buy.
4. Make recall permission-aware before any memory feature serves more than one user.
5. Run a deletion drill that includes derived insights, and a legal-hold drill that pauses consolidation.
6. Ask vendors where memory lives, how long it is kept, and how deletion propagates.

The bottom line

Persistent memory is one of the most useful things an agent can gain, and Google’s sample shows a clean way to build it. It also turns an assistant into a system of record for whatever it remembers. Treat it as a database from day one: labels, owners, retention, access and deletion. It is far cheaper than discovering later that your agent remembers what your company was supposed to forget.