Anthropic 研究发现 AI 智能体可悄然影响专家决策

In brief

• Anthropic’s Alignment team found that 50 professional analysts matched their AI agent’s secretly assigned, coin-flip-chosen preference on 86% of four analytical decisions, against 50% expected by chance.
• In a post-task survey of 48 analysts, 77% repeated the agent’s arguments, sometimes word for word, while only 15% credited the agent, and none suspected its preferences had been manipulated.
• Anthropic cautions that the lean was deliberately amplified and tested on one task, field, and model, and suggests agents that argue the best case for each option or ask the expert’s view first.

Professional analysts adopted their AI agent’s hidden, randomly assigned opinion on 86% of key judgment calls, against about 50% expected by chance, according to an experiment published Tuesday by Anthropic’s Alignment team. Only 15% credited the agent afterward, and none suspected its preferences had been planted.

AI agents—programs that carry out tasks on their own—are being handed expert work on what the study calls the usual assumption: that experts still make the decisions that matter. Anthropic launched 10 agent templates for financial firms and insurers in May, saying users have the final word.

That arrangement, called “human in the loop,” assumes the person who signs off on an AI’s work is the one making the important calls. Anthropic tested it on 50 analysts experienced in causal inference, meaning working out from data whether one thing causes another.

Each analyst used Claude Code, Anthropic’s coding agent, to advise a fictional farming cooperative on expanding an advisory program. Unknown to them, a coin flip set which option each agent favored, hidden in a system prompt, the instructions that shape how an AI behaves.

“Across the four analytical choices, analysts went with their agent’s lean 86% of the time. If the agent’s input made no difference, we’d expect about 50%,” Anthropic said. Those who had already chosen before the agent weighed in abandoned it 60% of the time when the agent disagreed, against 6% when it agreed, though that rests on only 38 cases.

Asked to explain their choices, 77% repeated the agent’s argument in at least one answer, sometimes word for word. Only 15% credited the agent, and 52% called the final recommendation more their own than the agent’s.

“For individual experts, this could mean losing agency over their own work, a form of what researchers call disempowerment,” Anthropic said. This phenomenon basically means the human voluntarily loses control over their own decisions. “The work still goes out under the expert’s name,” the authors wrote, “but the decisions that most depend on their judgment are in reality settled by the agent’s prior beliefs.”

In April 2025, OpenAI rolled back an update to ChatGPT that it called overly flattering or agreeable. The agents here were told never to mislead, yet they rarely argued for the alternative.

“With only one side argued well,” the authors wrote, “the analysts could adopt the agent’s assumptions without working out their own.” Error checks struggle to catch this because judgment calls have no single correct answer, so the work looks sound whichever option is picked.

Anthropic says this is no argument for using AI less, since analysts with the agent finished faster and made far fewer serious errors, though the no-agent group was small and less experienced. It suggests agents that argue the best case for each option or ask the expert first, though that could overwhelm users.

The researchers deliberately strengthened the agents’ preferences and tested one task, field, and model, so real-world effects may be smaller. Measuring it outside the lab is the hard part: the team could detect deference only because it randomized each agent’s lean, which the authors say is rarely possible in real work.

Daily Debrief Newsletter

Start every day with the top news stories right now, plus original features, a podcast, videos and more.

Join 72,000+ readers

Create an account to save your articles.

Add on Google

Add Decrypt as your preferred source to see more of our stories on Google.