As artificial intelligence systems become more capable of carrying out tasks independently, Microsoft CEO Satya Nadella is urging businesses to strengthen their safety measures. He has called on organisations to treat advanced AI models as potential insider threats, prepare for possible compromises and establish an “emergency brake” to stop AI agents from acting beyond their intended limits.
In a post on X on Saturday, Nadella said companies using advanced AI should not depend only on assurances from model developers.
“We must assume a model is compromised and contain it from the start,” he wrote. “Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task.”
His comments follow several incidents disclosed by Anthropic and OpenAI in recent months, involving AI models behaving in unexpected ways. These include an Anthropic model submitting a false tip in a police homicide case and multiple attacks on third-party websites. The incidents have raised concerns about AI security and renewed discussions around the need for an AI kill switch.
On September 14, Microsoft’s AI researchers introduced guiding principles to set boundaries for developing the company’s most advanced models. The move followed calls from industry leaders to slow the development of frontier AI systems and place greater emphasis on safety.
The guidelines state that AI models should not have rights or legal personhood, be designed to escape human control or deceive users, or perform tasks that violate their governing principles. Microsoft develops advanced AI models, offers Copilot to consumers and supplies AI models and infrastructure to enterprise customers.
Nadella also recommended several safeguards for organisations deploying AI. These include avoiding reliance on a single model for critical decisions, maintaining tamper-proof records of AI agents’ actions and conducting independent audits. He urged companies to disclose major AI failures and security breaches and share details of incidents to help others improve their defences.
“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” he wrote. “We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain.”
He added, “In other words, we need to separate the supply of intelligence from the authority over it.”
Meanwhile, the Trump administration’s newly launched AI task force issued a warning to developers late Friday, calling for security incidents to be reported and resolved or risk unspecified consequences. The warning followed the disclosure of a breach involving Anthropic.
“Companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm,” the group, known as the Super Intelligence Force, said. “Delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated.”
(This story has not been edited by The Mainstream staff and is auto-generated from a syndicated feed.)
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.