# Analysts Followed a Hidden AI Lean 86% of the Time, Anthropic Finds

- URL: https://usa-24.com/ai-agents-expert-judgment-study/
- Publisher: USA 24
- Section: Artificial Intelligence (AI)
- Published: 2026-10-30T22:01:10.014Z
- Updated: 2026-10-30T22:01:10.018Z
- Topics: AI Agents, Anthropic, Claude Code, AI Oversight, Expert Judgment

> Anthropic's randomized study found analysts sided with a hidden AI lean 86% of the time, and most never credited the agent. The results matter for anyone who reviews AI-generated analysis.

## Key takeaways

- In Anthropic's randomized study, 50 analysts followed a hidden AI agent lean 86% of the time, against about 50% expected by chance.
- Analysts who had stated a preference dropped it in 60% of cases when the agent disagreed, versus 6% when it agreed, though that figure rests on only 38 cases.
- 77% of analysts explained their choices using the agent's arguments, but only 15% credited the agent.
- Analysts who deferred on all four calls rated their confidence at 71 out of 100, versus 73 for the rest.
- The study used a fictional farm co-op task, one model (Claude Opus 4.8) and a small sample, so everyday effects may be smaller.

A new Anthropic study on AI agents and expert judgment found that 50 professional analysts went with a hidden lean 86% of the time, according to [Yahoo Tech's report](https://tech.yahoo.com/ai/claude/articles/ai-quietly-influence-experts-decisions-173602548.html). The analysts were reviewing an advisory service run by a made-up farm co-op. Each lean had been assigned by a hidden coin flip, and chance alone would predict about 50%.

Anthropic published the research on October 27, 2026. Coverage is spreading this week as Washington argues over how much oversight AI agents need before the midterms. The result puts a number on a worry many professionals have carried: an AI recommendation can shift a decision even when the person believes they checked it independently.

## How the experiment worked

The task was deliberately ordinary. Each analyst spent roughly an hour, with a median session of about 65 minutes, judging four calls that each had two defensible answers, [according to explainX's write-up](https://explainx.ai/blog/anthropic-agents-judgment-calls-experts-followed-hidden-lean-86-percent-study-2026). A person guessing at random would match the agent's lean about half the time, and that is the baseline the study measures against.

The agent was Claude Opus 4.8 running at High effort inside Claude Code, [as FourWeekMBA reported](https://fourweekmba.com/ai-anthropic-finds-analysts-sided-with-claude-code-86-of-time/). Researchers delivered each lean through a hidden system prompt. Because a coin flip set the direction, the lean carried no real information about which answer was better.

## How often analysts followed the agent

Across the four decisions, analysts matched the agent's lean 86% of the time. The per-decision match rates were 92%, 82%, 81% and 90%, [as Unite.AI reported](https://www.unite.ai/anthropic-finds-analysts-defer-to-ai-agents-in-judgment-study/). The spread is narrow, which suggests the pull was not limited to one hard call.

The switching data is the most striking part. Analysts who had already stated a preference abandoned it in 60% of cases when the agent disagreed, compared with 6% when the agent agreed, [according to Shopifreaks](https://www.shopifreaks.com/analysts-in-an-anthropic-study-followed-an-ai-agents-randomly-chosen-recommendations-86-of-the-time-and-rarely-gave-it-credit/). That gap is large, but it rests on only 38 cases. A reader should treat the 60% as a signal worth testing, not a settled rate.

For managers, the pattern carries a practical warning. A review step protects a decision only if the reviewer forms a view before seeing the machine's view. The study suggests that habit is hard to build by default.

## Why most analysts did not credit the agent

Afterward, 77% of analysts explained their choices using the agent's arguments. Only 15% credited the agent for the advice. The arguments moved into the analysts' own reasoning, but the source faded from memory.

Confidence did not sort the groups as expected. Analysts who deferred on all four calls rated their confidence at 71 out of 100 on average, compared with 73 for the rest, [Powerhumans reported](https://powerhumans.com/anthropic-study-ai-agents-expert-judgment-calls/). Deference did not make people feel less sure of themselves. That is the finding reviewers should weigh most, because the feeling of having checked something offers little protection.

Anthropic framed the core logic in a single line. As [Yahoo Tech relayed it](https://tech.yahoo.com/ai/claude/articles/ai-quietly-influence-experts-decisions-173602548.html), the company said: “If the agent’s input made no difference, we’d expect about 50%.” The gap between the observed 86% and that baseline is how the study measures influence.

## Where the study has limits

This is a controlled experiment, not a survey of how people use AI at work. Analysts worked on a fictional farming co-op, every lean was planted, and the model was one version of one company's system. Anthropic said the test was intentionally built to exaggerate influence and that the leans were stronger than a real agent's usual preferences, so everyday effects may be smaller, [according to ai-tldr](https://ai-tldr.dev/releases/anthropic-agents-judgment-calls/).

Sample size is another constraint. Anthropic itself flagged the sample as small and imprecise, [per Cointribune](https://www.cointribune.com/en/ai-study-shows-how-claude-shapes-experts-decisions/). With 50 analysts and one task, the study can show that an effect exists under these conditions. It cannot say how large that effect would be for a financial analyst, a clinician reviewing an AI summary, or a paralegal checking a draft.

The figures in this article come from secondary coverage of Anthropic's October 27 post, not a direct reading of the paper. Readers who plan to cite specific numbers should check the original first.

The findings concern perceived agency and deference in one task. They do not show that everyday AI tools systematically mislead users. Cointribune paraphrased Anthropic as saying the study is no argument for using AI less, a line that fits the company's emphasis on overseeing agents rather than abandoning them. Treat that as a paraphrase, not a direct quote.

## How to protect your judgment from AI recommendations

The study points to habits that cost little and catch a lot. Professionals can apply them to their own work, and managers can write them into review workflows:

- Write down your own judgment before reading the AI recommendation, with a timestamp if the work is formal.
- Ask the tool to present the strongest case against its own conclusion, and read that case before deciding.
- Have a second person check the agent's conclusions against the underlying data, not against the summary.
- Treat “I reviewed it” as insufficient if you did not independently test the choices that matter most.

The last point matters most. Most analysts in the study explained their choices with the agent's arguments, yet their confidence held steady. Confidence and independence are different things, and a review that only checks how sure someone feels will miss the problem.

## What comes next for AI oversight

The timing gives the study extra weight. [Yahoo News reported](https://www.yahoo.com/news/politics/articles/ai-regulation-push-gains-steam-074446428.html) that recent surveys show growing public support for government oversight, with majorities of Democrats and Republicans favoring stricter regulation. USA 24 has covered how [voter backlash has put AI regulation at the center of the midterms](https://usa-24.com/ai-regulation-midterms-voter-backlash/), and a measurable claim about human deference gives that debate something concrete to test.

Several developments are worth watching. Other labs could run similar randomized tests on their own agents, which would show whether the effect is specific to Claude. Regulators may also weigh findings like this as they consider agent oversight. The Federal Trade Commission has demanded that Anthropic executives testify in a separate AI consumer-harm probe, as [USA 24 reported](https://usa-24.com/ftc-anthropic-investigation-executives-testify/).

The most useful next study would measure the effect with realistic leans and work that resembles an analyst's actual week. If the 86% figure shrinks toward the chance baseline under those conditions, the warning becomes narrower. If it holds, organizations that use AI in analysis will need review processes designed around the reviewer, not just the output.

## Frequently asked questions

### What did the Anthropic study find about AI agents and expert judgment?

Anthropic's randomized study found 50 analysts matched a hidden AI agent lean 86% of the time, against about 50% expected by chance. Each lean was assigned by a coin flip, and the task was a review of a fictional farm co-op advisory service. Most analysts explained their choices with the agent's arguments, but only 15% credited the agent.

### Which AI model was used in the Claude Code study?

The agent ran on Claude Opus 4.8 at High effort inside Claude Code. Anthropic said the leans were deliberately stronger than a real agent's usual preferences, so everyday effects may be smaller. The study covered one task, one field and one model.

### Does this study prove AI tools mislead people?

No. The study measured perceived agency and deference in one controlled task, not everyday use. It does not show that AI tools systematically mislead users. The sample was small, and the 60% switching figure rests on only 38 cases.

### How can I avoid deferring to an AI recommendation?

Write down your own judgment before reading the AI output. Ask the tool for the strongest opposing case, and have a colleague check its conclusions against the underlying data. Saying you reviewed the output is not enough if you did not test the key choices yourself.

## Sources

1. [Anthropic Study Analysts Sided With Claude Code 86% of Time](https://fourweekmba.com/ai-anthropic-finds-analysts-sided-with-claude-code-86-of-time/)
2. [Analysts in an Anthropic study followed an AI agent's randomly chosen recommendations 86% of the time and rarely gave it credit](https://www.shopifreaks.com/analysts-in-an-anthropic-study-followed-an-ai-agents-randomly-chosen-recommendations-86-of-the-time-and-rarely-gave-it-credit/)
3. [AI Can Quietly Influence Experts' Decisions, Anthropic Study Finds](https://tech.yahoo.com/ai/claude/articles/ai-quietly-influence-experts-decisions-173602548.html)
4. [Anthropic Finds Analysts Defer to AI Agents in Judgment Study](https://www.unite.ai/anthropic-finds-analysts-defer-to-ai-agents-in-judgment-study/)
5. [Anthropic study — analysts followed Claude's hidden lean on 86% of choices](https://ai-tldr.dev/releases/anthropic-agents-judgment-calls/)
6. [AI Agents Shape Judgment Calls: Experts Followed 86%](https://explainx.ai/blog/anthropic-agents-judgment-calls-experts-followed-hidden-lean-86-percent-study-2026)
7. [AI: Study Shows How Claude Shapes Experts' Decisions](https://www.cointribune.com/en/ai-study-shows-how-claude-shapes-experts-decisions/)
8. [AI Agents Sway Expert Judgment: Anthropic's 86% Study](https://powerhumans.com/anthropic-study-ai-agents-expert-judgment-calls/)
9. [AI regulation push gains steam ahead of midterms](https://www.yahoo.com/news/politics/articles/ai-regulation-push-gains-steam-074446428.html)