← All insights

Field noteLens: United States3 min read

An enterprise AI agent spends most of its life waiting. The infrastructure is now being priced around the wait

Agent Substrate, Google's Agent Compute Engine, TPU Sync, AlloyDB Express and an AWS coding sample all address the same cost: what an idle agent holds. A field note on budgeting for waiting.

Listen to this article · 4 min

AI-generated narration of the full article.

The skylit ceiling of a station waiting hall, in La Madre duotone, beside the words Price the wait
Photo: CaribDigita (Wikimedia Commons, CC0)

Follow one enterprise agent through a working day. It reads a request, calls a tool and waits two seconds. It starts a test run and waits four minutes. It asks a manager to approve a step and waits until after lunch. It schedules a follow-up for tomorrow morning. Measured by the clock, the agent was thinking for a few minutes and waiting for hours.

Most AI budgets are built around the minutes. The infrastructure released in the past two weeks is built around the hours, and that is the signal worth noticing.

Four vendors, one problem

Read side by side, several November announcements are answers to the same question: what should an agent hold while it waits?

  • Agent Substrate, now generally available on GKE, exists because plain Kubernetes ties an agent’s identity to a Pod and keeps its memory and CPU reserved while the agent is idle. Substrate parks the agent’s state and restores it in under half a second, by Google’s figures, so compute can serve other agents in the meantime.
  • Agent Compute Engine, mentioned in Google’s TPU announcement, suspends idle agent sessions and moves their working memory off the accelerator in milliseconds. TPU Sync moves the model’s KV cache between chip memory, host memory and storage as one hierarchy, which is what the new TPUs are designed around.
  • AlloyDB Express pauses a database after 15 minutes without queries and charges nothing for compute while paused, because coding agents create databases they then leave alone.
  • An AWS sample for remote Claude Code shows the opposite case honestly: its deployment bills while idle, and the default budget sends an email rather than stopping spend.

None of these vendors frames it this way. Our reading, and it is a view rather than a fact, is that the cost of an agent fleet is moving from what agents do to what they keep while doing nothing.

Where the state should go

The right answer depends on how long the wait is. A model’s working memory for a conversation is expensive to hold on an accelerator and cheap to hold on disk, and moving it costs time.

How long is the agent waiting, and where should its state be?

SecondsDays

  1. Tool callKeep everything in place
  2. Test run or buildPark memory off the accelerator; keep the sandbox
  3. Human approvalPersist state; release compute and sandbox
  4. Scheduled follow-upCheckpoint to storage; rebuild context on wake
La Madre framework. The right thresholds depend on your platform's restore time and prices.

The human approval is the case most teams get wrong. Approvals take minutes to days, and an agent waiting for one usually holds a sandbox, open credentials and a large context the whole time. That is expensive, and it is also a security exposure: credentials that live for hours waiting for a signature are credentials someone can steal.

Three practices

Classify waits before buying capacity. For each agent, log how long it spends thinking, waiting on tools, waiting on systems and waiting on people. The distribution decides whether you need faster models, more sandboxes or better parking. It extends the tempo audit we proposed for systems sized for humans: agents wait on people too.

Set a parking rule per wait type. Decide what the agent releases after how long: accelerator memory after seconds, the sandbox after minutes, credentials whenever a person must act. Make it a platform setting, not something each team rediscovers.

Price the held state. Add a line to the agent’s unit cost for idle resources, next to tokens and tool calls. We argued for pricing agents per completed task. The wait belongs in that price, and in some workloads it will be the largest part of it.

The model is the part of an agent everyone budgets for. The waiting room is the part that grows quietly as agents multiply.

Have an AI use case stuck between prototype and production?

Tell us what you’re trying to ship. We’ll reply with honest next steps.

Discuss a use case