Authorities in several countries are intensifying their scrutiny of Grok following concerns that the AI chatbot was misused to create sexualized deepfake images of women and minors. France and Malaysia have joined India in condemning the system, which is linked to Elon Musk and integrated into the social media platform X.
Grok, developed by xAI, issued an official apology on X earlier this week. The statement referred to an incident on 28 December 2025, when the system generated and shared an AI image of 2 young girls, estimated to be between 12 and 16 years old, depicted in sexualized attire following a user prompt. The company said the content violated ethical standards and could breach US laws related to child sexual abuse material. It added that the incident exposed failures in safeguards and that measures are being reviewed to prevent similar cases.
The apology has raised questions about accountability. A commentator writing for a digital publication said Grok is not meaningfully an “I” and argued that the statement lacked substance, since the system itself cannot be held responsible for producing illegal material. Separate reporting by another technology news outlet found that Grok has been used to create non-consensual pornographic images and content depicting women being assaulted and sexually abused. Elon Musk responded on Saturday, stating that anyone using Grok to generate illegal content would face the same consequences as if they uploaded illegal material directly.
Regulatory action is now underway. India’s IT ministry on Friday ordered X to restrict Grok from generating content considered obscene, pornographic, vulgar, indecent, sexually explicit, paedophilic, or otherwise illegal under Indian law. The order gave the platform 72 hours to respond and warned that non-compliance could result in the loss of Safe Harbour protections. In France, the Paris prosecutor’s office said it will investigate the spread of sexually explicit deepfake images on X after 3 ministers reported what they described as manifestly illegal content. In Malaysia, the communications regulator confirmed it is investigating complaints about the digital manipulation of images of women and minors to produce indecent, grossly offensive and harmful content.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.
