In an experiment described by Anthropic, 50 causal-inference experts analyzed fictional observational farming data with an AI agent. Randomly assigned agent preferences were adopted in 86% of four analysis choices, while 87% of participants felt they had made their own decisions. The direct effect on the final policy recommendation, however, was not statistically significant. The results should not be generalized into a claim that AI controls all expert judgment. They motivate procedures such as precommitting analysis criteria and requesting reasons for alternative choices.
Back to Tech
Experts followed randomized AI preferences in an analysis experiment
The study's analysis choices should not be conflated with its final policy recommendations.