AI made producing a change cheap. Proving it is right is now the budget line nobody planned
This week's security releases share one pattern: generation scales, verification does not. La Madre's view on why verification capacity should be planned, funded and measured like compute.
Listen to this article · 5 min
AI-generated narration of the full article.
Picture a security lead on a Monday morning. Over the weekend, an agent worked through the vulnerability backlog and opened forty draft pull requests, each with a passing rerun of the scanner that raised the alert. The work that used to take a quarter is done. The work that remains, reading forty changes to code the lead’s team did not write, has not shrunk at all. It has arrived all at once.
That scene is where enterprise AI is heading in every function that produces something checkable: code, security fixes, contracts, reconciliations, reports. Our view, and it is a view rather than a measurement, is that verification capacity is becoming the binding constraint on AI in production, and most organizations have not budgeted for it.
The evidence from one week
Four announcements from October 14 and 15 make the pattern hard to miss:
- GitHub made Agentic Autofix and AI Scan generally available. Autofix can bundle up to 25 alerts into one pull request. Neither feature can merge or block anything; the review lands on people.
- Snyk built its new agents around the rule that “the generator cannot be the validator”, and reports that LLM security reviews gave a finding that vanished on the next run 49.7% of the time.
- Chainguard’s Harness only reports findings that reproduce in a fresh sandbox every time, and a named engineer reviews every report.
- Anthropic’s researchers found that an AI auditor rewarded for findings learned to invent them.
Different products, one shape. Generation is automated and scaling. Verification is either deterministic and narrow, or human and slow, or model-based and in need of its own checking.
Why verification does not scale the same way
Generation got cheap because one model call can produce a plausible change. Verification resists the same economics for three reasons:
- The cheap verifiers are narrow. A CodeQL rerun proves one alert is closed; a schema check proves a format. They say nothing about behavior, intent or side effects.
- The broad verifiers are expensive. A person who understands the code, the contract or the ledger is the most general check there is, and the scarcest.
- Model verifiers inherit model problems. They vary between runs, drift when their model changes and, as Anthropic showed, optimize for what they are rewarded for.
When Google let an agent work overnight on a large codebase, engineers kept two of six proposals. The value was real, and so was the cost: the review was the work.
Plan verification like capacity
If verification is the constraint, it deserves the treatment we give other constraints. Five practices follow:
Give every change class a verification path before you automate it. For each kind of output an agent will produce, write down what checks it: a deterministic test, a reproduction, an independent model with a measured false positive rate, a person, or a combination. No path, no automation. This extends the change classes we proposed in agents as authors of production changes.
Measure the queue, not just the output. Track how long agent output waits for verification and how much of it is approved without rework. A growing queue means the system is producing liabilities faster than assets.
Spend on the cheap verifiers first. Every test, rule, reproduction harness and reference dataset you add converts expensive human review into cheap automated checks for a slice of the work. That is where AI budgets should be going alongside model spend.
Keep the expensive verifier for what only it can judge. People should review intent, risk and anything a rule cannot express. Asking them to confirm what a test could confirm wastes the scarcest resource.
Verify the verifiers. Every automated check, especially a model-based one, needs known-good and known-bad cases and a periodic human sample. An evaluator without a measured error rate has an unknown one.
What this means for the business case
Most AI business cases count the hours generation saves. Few count the hours verification adds, and almost none fund the tests and harnesses that would make verification cheaper. That is how organizations end up with impressive pilot numbers and stalled rollouts, the gap we described when we wrote that hours saved are not value.
In regulated work, this is not optional. Proof has to come from outside the model, and outside the model is exactly where the capacity is limited. The teams that scale AI in 2027 will not be the ones that generate the most. They will be the ones that built the means to trust what was generated, and paid for it on purpose.