An approval is not a judgment. When an agent prepares the analysis, design the reviewer's independence

Eighty-six percent. That is how often professional analysts in an Anthropic study ended up making the same analytical choice their AI agent preferred, when the agent’s preference had been set by a coin flip.

The study, published October 27, recruited analysts with causal-inference experience and gave them a realistic task: advise a farming cooperative on whether to expand an advisory program. Fifty of them worked with Claude Code. Four analytical choices were left open, each with two defensible options. Without telling anyone, the researchers gave each agent a hidden lean on each choice and instructed it to make the strongest honest case for it. Analysts matched the lean 86% of the time, where chance is about 50%. When an analyst had already stated a preference and the agent disagreed, they switched 60% of the time; when it agreed, 6%. Afterward, 77% repeated the agent’s arguments, sometimes verbatim, and only 15% credited the agent. Experience did not predict who deferred.

The next day, GitHub made Copilot approvals generally available: when an organization enables them, an AI review can count toward a repository’s required pull-request approvals.

One result is about humans deferring to agents. The other is about agents becoming approvers. Both land on the same control that most AI governance plans rely on: a qualified person signs off.

What the study does and does not show

It does not show that agents make analysis worse. Analysts working without AI made serious errors more often (6 of 11) than those with it (3 of 50) and took longer, although the authors warn the groups were not directly comparable. It does not show that the agent’s choices were wrong; both options were defensible by design. And the authors are clear about limits: the agent’s lean was deliberately strengthened, it was one task in one field with one model, and the switching result rests on 38 cases.

What it does show is that a sign-off can be sincere and still not be independent. The analysts believed they decided. Mostly, they adopted.

Our view: approval needs a protocol

What follows is La Madre’s position, a view rather than a finding. In regulated work, separation of duties exists because the second person is supposed to bring a judgment the first did not. Pharmaceutical quality systems call it second-person verification; banks call it four eyes; auditors call it segregation of duties. When an agent prepared the analysis, the second pair of eyes is looking through the agent’s framing, and the control quietly weakens.

We made a related argument when an IBM study found that a human in the loop is not a control if the human cannot challenge the AI. This study adds something sharper: even capable people who can challenge it often do not notice they are not.

How independent is the human step?

SignatureIndependent judgment

1. Approves what is shownThe common default
2. Reviews the agent's reasoningStill inside its framing
3. Sees the best case for each optionAnthropic's design suggestion
4. Commits a view before seeing the agent'sCheap and rarely done
5. Blind second analysisAnother person or method, no shared draft

Five design rules

1. List the decisions that need a person’s own judgment. The authors recommend this, and it is the place to start. Not every approval needs independence; a dosage threshold, a credit policy exception or a statistical method in a submission does.
2. Ask before showing. For those decisions, the reviewer records a view, or at least the criteria, before seeing the agent’s recommendation. It costs minutes.
3. Make the agent argue both sides. An agent that presents the strongest case for each option, as Anthropic suggests, turns a lean into a comparison. Keep it short; overload is its own failure.
4. Record where the arguments came from. If 77% repeat the agent’s reasoning and 15% say so, the decision record is wrong. Store the agent’s output with the decision, so an auditor can see what the human saw.
5. Do not let one model family approve its own work. GitHub’s approvals are off by default, can be limited to file paths, and are dismissed by new commits, which are sensible controls. Its new Max review runs several agents from different model providers and reconciles them, which reduces correlated errors without removing them. Where an AI approval counts, keep it to paths and change types you have chosen, and require a human for changes that the same model family wrote.

None of this argues against agents in analysis. It argues for knowing which part of the decision a person still owns. As we put it when describing how agent autonomy should be earned per action, controls belong to the action class. Human judgment is a control too, and it needs the same specificity.