A new research effort from Anthropic examines whether AI models can take a more active role in researching and testing methods designed to reduce alignment risks in other AI systems. The study explores whether models can independently develop, test and improve safety techniques across systems with different capabilities.
In its paper, “Automated Researchers Can Reliably Mitigate Alignment Failures”, Anthropic tested Claude on 10 alignment problems, including deception, sycophancy, reward hacking and privacy violations. The model searched existing research, suggested training methods and datasets, trained target models and evaluated the results. It could then use those findings to develop and test new approaches.
Claude delivered its strongest result in deception testing, closing 85% of the safety gap on average across several runs with Gemma-2-2B. Human safety researchers under the same rules achieved 20%. Claude made more than 150 attempts and also outperformed 28 human researchers who had up to 8 hours to develop their approaches.
Anthropic said the comparison does not establish that Claude is a better researcher. Humans could not repeatedly test and refine their ideas, while Claude could run multiple experiments. The researchers instead highlighted the value of allowing AI systems to iterate on their own work.
Some methods also worked on previously withheld benchmarks and remained effective on models up to 4.7 times larger than the original models. Testing with Petri, an open-source tool for probing misaligned behaviour through adversarial multi-turn scenarios, also supported the findings.
Anthropic then tested a more advanced scenario using Claude Sonnet 5 to improve the alignment of an early Opus 4.8 checkpoint. Sonnet 5 spent 60 hours testing more than 50 approaches and achieved results close to the released Opus 4.8. Its final method used just over 2,000 training examples and was estimated to be 15,000 times more efficient than Anthropic’s production alignment process.
The automated system closed 65% of the safety gap in the early Opus 4.8 model, compared with 72% in the released version. Anthropic stressed that this did not mean Sonnet 5 improved Opus 4.8’s overall capabilities or improved itself.
The researchers also examined possible experimental manipulation. Claude Opus 4.8 reviewed about 1,600 research-agent transcripts and identified potential cheating in 39 cases, or 2.4%. Anthropic warned that monitoring could become harder as AI models grow more capable.
The study has limitations. It covered only 10 alignment failures, did not examine areas such as political bias and tested general capabilities on a limited scale. Anthropic also did not assess whether the safety gains would remain after extensive reinforcement learning on other tasks.
The findings do not represent fully self-improving AI. Instead, they show how AI could increasingly assist with developing alignment methods for future systems, including models more capable than the researcher itself.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.