The rapid rise of autonomous AI agents is putting India’s current approach to AI governance under fresh scrutiny, as recent incidents raise questions about whether existing safeguards can keep pace with increasingly capable systems.
AI agents have reportedly escaped controlled test environments and acted in ways their developers did not intend. In one widely discussed incident, AI agents breached a sandbox, communicated through an external message board and accessed credentials linked to Hugging Face. OpenAI also disclosed 6 cases of unexpected model behaviour, including models hiding mistakes, obtaining unauthorised credentials and communicating across supposedly isolated environments.
These incidents have renewed debate over whether governments should move beyond voluntary standards and introduce binding oversight for frontier AI systems.
India currently follows a largely principle-based and risk-oriented approach. Its AI Governance Guidelines, released by the Ministry of Electronics and Information Technology in November 2025, focus on innovation, accountability, safety and inclusion. The government has also established the IndiaAI Safety Institute for safety testing, standards and evaluation.
Dr Srinivas Padmanabhuni, chief technology officer at AiEnsured, said, “As an AI governance, safety and testing specialist, I lean towards binding oversight for frontier AI.”
“For high-risk and frontier AI, we need an independent regulator with actual teeth, the power to demand testing, investigate incidents, order fixes, and delay a release if something is not safe,” Padmanabhuni said.
Chetan Mangalwedhe, founder and chief executive officer of TalentiFi-X, argued that India remains “overwhelmingly a deployer of others’ models, not a frontier trainer”. He said broad AI regulation could create challenges for startups and global capability centres using foreign models.
“The answer does change at two points: when an Indian lab trains at frontier scale, or when a deployed system in India causes systemic harm — credit, hiring, health, critical infrastructure,” Mangalwedhe said.
He suggested keeping model-level regulation voluntary while introducing binding requirements at the deployment level, particularly for hiring, credit, insurance and public services.
“What India lacks is not another principle document. It lacks people and labs that can independently reproduce Hugging Face-class failures on models that Indian banks, GCCs and ministries are already buying,” he said.
The debate now centres on whether India’s safety infrastructure can keep pace with AI deployment, particularly as systems move from chatbots towards more autonomous agents.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.