Anthropic, the creator of Claude, asked 50 professional analysts to advise a fictional farming cooperative. The researchers wanted to understand whether experts would make independent decisions when using AI assistants.
Participants assessed whether a fictional advisory programme for the cooperative’s members was effective and whether it should be expanded. They first made 4 choices between 2 reasonable options before giving a final policy recommendation.
Without the participants’ knowledge, researchers instructed each AI assistant to favour a specific option. This allowed them to measure how much the technology influenced the analysts’ decisions.
According to the study, analysts selected options that matched their AI assistant’s preference in 86% of cases. Without such influence, the researchers expected a 50/50 split. Among participants who already had a preference, 60% changed their decision when the AI assistant disagreed with them.
The study also found that 77% of analysts used arguments from their AI assistants to explain their decisions. However, only 15% acknowledged AI as the source of their reasoning.
Anthropic noted that the research had limitations. It involved only 1 task, and the AI assistants had been deliberately instructed to favour particular choices.
Researchers Amaç Herdagdelen, Hailey Joren and Alex Tamkin said the tendency to follow AI recommendations may be “hard to identify, even for the experts themselves”.
They warned that AI systems’ “priors” could influence important policy decisions, including those related to AI safety.
“If human judgement is to remain a meaningful check as more work is performed with AI, oversight of these decisions has to look at how they get made: Surfacing key decision points as the work proceeds and helping experts articulate the assumptions and values they would otherwise leave unstated,” the researchers wrote.
They added, “A process that lets experts discover their preferences may matter for empowerment as much as the decision itself.”
Anthropic is reportedly preparing for an IPO valued at up to $2 trillion. The company and its competitors face growing public debate over the risks of advanced AI. While supporters expect major productivity gains, rapid development has raised concerns about job losses, cyberattacks and other societal impacts.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.