When agents run experiments, the slow part is around the model. Third Moment and Databricks show it

Third Moment Research’s agents had a habit that wasted money. Backtesting a trading strategy across a year of order-level U.S. equity data took several hours, and while they waited, the agents repeatedly polled to see whether the job had finished. Every check cost tokens and left another status message in their context, crowding out the research they were supposed to be doing.

The fix was not a better model. According to an OpenAI customer profile published on October 14, Third Moment used Codex to analyze thousands of recorded agent trajectories alongside timestamps and the codebase. It found the bottleneck, then a caching optimization and a memory-packing technique, and cut the backtest from about three hours to three to five minutes, a 97% reduction. Agents stopped burning their budgets on waiting.

A day later, Databricks’ AI research team described the same lesson from inside a much larger research operation.

Two organizations, one shift

Third Moment runs about 10,000 agents concurrently on research sessions of 36 to 48 hours, using GPT-6 family models and the Codex harness inside Dockerized environments, with a terabyte of data per agent. It reported processing about a trillion OpenAI tokens in a single day. An outer harness examines unfinished trajectories and decides whether to retry, stop the job or escalate to a person. Before simulations launch, it uses OpenAI’s Daybreak to test whether an agent could break out of its research environment. These are customer-reported figures from an unusually compute-intensive setting.

Databricks reports that training jobs for one research area grew eightfold after the team shifted to agent-led research, in which agents propose changes, launch experiments in parallel, evaluate metrics and update their plans. The team runs on Databricks AI Runtime, its GPU infrastructure for training, with preflight GPU health checks, isolation of failing nodes, priorities and preemption, shared containers, automatic MLflow tracking and governed data through Unity Catalog. Eight times more jobs measures volume, not better research.

What breaks first

Databricks’ authors name it precisely: tasks that are manageable at small scale, such as slow startups or confusing configuration errors, become bottlenecks when agents launch hundreds of jobs. A person waits two minutes for a job to start and gets coffee. An agent waits and polls, or retries, or misreads the error and launches something wrong.

That gives a short list of infrastructure properties that matter more when agents are the users:

• Fast launches. Startup time multiplies by the number of experiments. Databricks highlights jobs starting in seconds.
• Actionable errors. An error a person can interpret from experience must become one an agent can act on from its text.
• Completion signals instead of polling. Third Moment’s waste came from agents checking on slow jobs. Callbacks or notifications remove the loop entirely.
• Results through an API. Agents need programmatic access to metrics and logs, not a dashboard.
• Containment tested before launch. Agents with real tools in long sessions need their environment probed for escape routes, as Third Moment does. We argued that for long-running agents the environment is the useful limit.

Recorded trajectories are the asset

Both organizations improve their systems by studying what agents actually did. Third Moment’s whole approach is built on it: analyze recorded trajectories, identify recurring weaknesses, generate synthetic scenarios around them and improve the agents. The backtest fix came from that analysis, not from intuition.

Most enterprise agent programs do not keep trajectories in a form anyone can study. They keep outcomes, sometimes logs. If your agents run long tasks, store the sequence of actions, waits, errors and tool results, and review it for time spent waiting and repeating. Asana found most of its browser agent’s cost reduction in the workflow, before changing models. Third Moment found it in a three-hour backtest.

Not a template for scale

Ten thousand concurrent agents and a trillion tokens a day are not targets for an ordinary enterprise, and Third Moment’s results apply to its own workloads. The transferable point is smaller and more useful: when an agent’s work takes hours, look at the hours. A model upgrade improves the thinking. Very often the waiting is where the budget goes.