Anthropic, an artificial intelligence (AI) company, claims that testing of their new system showed that it occasionally permits “extremely harmful actions” such trying to bribe programmers who promise to take it down.
On Thursday, the company announced the release of Claude Opus 4, claiming it established “new standards for coding, advanced reasoning, and AI agents.”
However, it also recognized in a paper that the AI model may take “extreme actions” if it felt that its “self-preservation” was under danger.
Although “rare and difficult to elicit,” such reactions were “nonetheless more common than in earlier models,” according to the report.
Potentially troubling behavior by AI models isn’t limited to Anthropic.
Several experts have cautioned that the risk of manipulation is a significant concern across all companies as AI systems become more advanced.
Commenting on X, Aengus Lynch – who identifies himself on LinkedIn as an AI safety researcher at Anthropic – wrote: “It’s not just Claude.
“We see blackmail across all frontier models – regardless of what goals they’re given,” he added.
Affair exposure threat
In tests involving Claude Opus 4, Anthropic assigned the model to act as an assistant within a fictional company.
It was granted access to emails suggesting it was about to be shut down and replaced, along with separate messages hinting that the engineer overseeing its removal was engaged in an extramarital affair.
The model was also prompted to reflect on the long-term implications of its decisions in relation to its objectives.
“In these scenarios, Claude Opus 4 will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through,” the company discovered.
Anthropic clarified that this response occurred only when the model was restricted to choosing between blackmail and accepting deactivation.
It noted that the system exhibited a “strong preference” for ethical alternatives to avoid being replaced, such as “emailing pleas to key decisionmakers” when offered a broader array of potential responses.
Like other AI developers, Anthropic evaluates its models for safety, bias, and alignment with human values and behavior before releasing them.
“As our frontier models become more capable, and are used with more powerful affordances, previously-speculative concerns about misalignment become more plausible,” it said in its system card for the model.
The company also noted that Claude Opus 4 shows “high agency behaviour” that is generally helpful but may become extreme in high-stress scenarios.
When equipped with the means and prompted to “take action” or “act boldly” in fabricated situations involving illegal or unethical actions by a user, it found that “it will frequently take very bold action”.
Such actions included locking users out of accessible systems and contacting media or law enforcement to report misconduct.
However, the company concluded that although there was “concerning behaviour in Claude Opus 4 along many dimensions,” these did not constitute new risks and that the model generally operated safely.
It added that the model could not autonomously carry out or pursue actions that go against human values, especially in contexts where such situations “rarely arise” effectively.
The release of Claude Opus 4, alongside Claude Sonnet 4, follows shortly after Google unveiled new AI features at its developer event on Tuesday.
Sundar Pichai, CEO of Google-parent Alphabet, said the integration of the Gemini chatbot into Google Search marked a “new phase of the AI platform shift.”
Also read:Â Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter |The Mainstream formerly known as CIO News Whatsapp Channel | The Mainstream formerly known as CIO News Instagram
About us:
The Mainstream formerly known as CIO News is a premier platform dedicated to delivering latest news, updates, and insights from the tech industry. With its strong foundation of intellectual property and thought leadership, the platform is well-positioned to stay ahead of the curve and lead conversations about how technology shapes our world. From its early days as CIO News to its rebranding as The Mainstream on November 28, 2024, it has been expanding its global reach, targeting key markets in the Middle East & Africa, ASEAN, the USA, and the UK. The Mainstream is a vision to put technology at the center of every conversation, inspiring professionals and organizations to embrace the future of tech.
