Databricks cut AI coding unit costs by up to 90% without capping its best developers. The gateway did the work

Databricks rolled out the frontier model Astra to every engineer, around 3,500 people. The model was clearly better on the hard problems, such as system design. Spend went up about 60%, and on everyday tasks, which the existing models already handled, the gain was not obvious.

That detail, from an interview Databricks published on October 22 with its co-founder Patrick Wendell, is the most useful number in a story mostly told through percentages. It shows the shape of the problem every company scaling coding agents now faces: the expensive model is worth it for some work, and most work is not that work.

How the bill got out of hand

Wendell describes a familiar sequence. To get reluctant engineers over the hump, Databricks removed every barrier: the most expensive model, as many tokens as anyone wanted. Then adoption took off. Because coding tools are now priced by consumption rather than per seat, one person can, in his words, “run a little loop on the most expensive model and go to lunch.” Costs grew exponentially while revenue did not, until they threatened to cancel the efficiency gains that justified the rollout.

The company says its token spend per head is now roughly flat while usage keeps rising, and that in some cases its unit costs fell by as much as 90%. These are internal figures from a company that sells the tool it used, and the post does not define the unit or the baseline. The levers behind them are more transferable than the numbers.

Four levers, in order of size

Move the default to a cheaper model when one is good enough. Wendell calls it boring and says it was the largest effect, 50% or more on its own. New, cheaper models arrive constantly; the saving only happens if someone evaluates them quickly on the company’s own code and switches the default in one place.

Route by task. Renaming a component does not need the model that designs systems. Databricks says its router cuts average task cost by more than 30% at roughly the quality of the most expensive model, and that it removes the burden of choosing from developers.

Show people their own spend. A regular personal breakdown with suggestions is worth about 10%, according to Databricks. Wendell mentions another company that prints the model and its cost on every pull request.

Cut token overhead. The harness around the model matters as much as the model. In Databricks’ own benchmark, the same model in two different harnesses differed by more than 2x in cost, and tuning the harness and caching saved close to 50%. That matches what we saw when Asana cut its browser agent’s cost 76x, mostly before it changed models.

Hard caps fail the people you most want to keep

The part of the interview most worth copying is about limits. A hard spend cap is the obvious first move, and Databricks found it a bad one. It stops people mid-task and costs them hours of work. The people who hit it are often the best users. And some of them respond by switching to a personal account, where the company can see nothing at all.

Databricks replaced the cap with friction that ramps up.

From open access to a hard stop: where Databricks now sits

Unlimited, most expensive modelHard cap

1. Where it startedEvery barrier removed to drive adoption
2. WarningThe developer sees the spend
3. ApprovalSomeone signs off to continue
4. DownshiftWork continues on a cheaper model
5. Cut-offLast resort

This matches the position we took when platforms began shipping spend guardrails: hard caps suit exploration, while production use needs budgets with alerts and a defined path past the limit. Downshifting is the elegant step. The developer keeps working; the company stops paying frontier prices for that session.

Price per task, measured on your own code

Two findings from Databricks’ internal benchmark, built from real pull requests and graded by running the tests rather than by an LLM judge, should change how teams compare models:

• An open model, GLM 5.2, tied Anthropic’s Opus 4.8 on quality at about a third less per task.
• A model that was 1.7 times cheaper per token cost more per task, because it used more tokens.

Token price is a poor proxy for cost. The unit that matters is the cost of an accepted change, which is the argument behind our framework for pricing inference per completed task, and behind the idea that the cheapest model is the one whose errors you can afford to catch. Databricks’ test-graded benchmark is a practical version of both.

The control layer is the precondition

Every lever above depends on one thing: all coding tool traffic passing through a single layer that can see it and steer it. Databricks routes Claude Code, Codex, Cursor and its own tools through one gateway, which it later turned into its Unity Gateway product. Wendell’s advice to a CTO is blunt: “Figure out where it’s going.” Most companies cannot say what they spend on AI by team or by model.

That layer is becoming a contractual question as well as a technical one, now that OpenAI wants to approve gateways that offer its models. The engineering case for having one is not in doubt. A company that cannot see its AI traffic cannot route it, budget it or switch it, and will keep paying for the Lamborghini on the trip to the corner shop.