Your first agent's most lasting result may be cleaner source data. Plan for it, and budget for it

There is a familiar moment in agent pilots. Someone shows a wrong answer on the screen, the room starts discussing the model, and then a person who knows the process traces the answer to a record nobody had filled in for two years. The model did exactly what it was asked. The data told it something false.

That moment usually ends with a workaround: a rule in the prompt, an exception in the retrieval, a note to “clean that up later”. Our view is that it should end differently. The defect the agent found is one of the most valuable things the pilot produced.

Agents read what people skip

People who work with a system for years learn which fields to ignore, which statuses are never updated and which customer has three records. Agents do not. They read records literally, at scale, every time. So they find, quickly and in public, the data debt that people had quietly worked around.

Four signals from a single day of publications make the pattern hard to miss:

• Microsoft’s infrastructure engineering team found that its AI-generated leadership report was inaccurate until it built a second agent to fix the planning data at the source. “The AI can only be as accurate as the data it has in front of it.”
• Microsoft’s internal guide to choosing between Agent Builder, Copilot Studio and Foundry states it as a lesson: data readiness drives agent quality, because an agent reflects the content it can access.
• Microsoft’s support agent lost users when it ignored what they had already tried. Its fix was partly about tone, and partly about carrying context, which is data that had not been captured or passed along.
• Google and Databricks both moved business definitions closer to agents, and conflicting definitions are the first thing such a layer exposes.

Two outputs, and most programs keep one

Every agent in its first months produces two things: answers, and a list of reasons some answers were wrong. Most programs keep the first and throw the second away in workarounds. We think the second list deserves to be treated as a deliverable in its own right.

Log every wrong answer by root cause. Model, retrieval, data, definition or process. Without that split, every defect looks like a model problem and gets a prompt fix. We made the case that retrieval needs its own measurement; data and definitions do too.

Route data defects to the owner of the system, not to the AI team. Microsoft’s hygiene agent does exactly this: it tells the record’s owner what is missing and how to fix it in the system of record. The AI team cannot fix a supplier master, and should not try.

Do not let the agent paper over gaps. An agent that fills a missing field with plausible text hides the defect and makes the next answer worse. Prefer agents that say “this field is empty” over agents that guess well.

Measure the repair. Completeness and consistency rates for the records the agent depends on, tracked over time. Those numbers outlast the agent, and they help every future tool, model and vendor that reads the same data.

Budget it on both sides

Data repair is real work by people who already have jobs, so it belongs in the business case as a cost. It also belongs there as value, because a cleaner record serves every process that reads it, not only the agent. That is the kind of value we argued hours saved cannot capture: it needs an owner and a metric, decided before go-live.

In regulated industries, this is less new than it sounds. Life sciences companies already inspect data integrity as a discipline, and banks already reconcile. An agent that exposes incomplete records is producing findings those teams know how to process. The risk is only in ignoring them.

Our expectation, and it is a view rather than a fact: the organizations that get the most from their first agents will not be the ones with the best first answers. They will be the ones that treated the wrong answers as an audit of their data, and fixed what it found.