Grok under scrutiny as France and Malaysia probe sexualised AI deepfakes

French and Malaysian authorities have launched investigations into Grok, the AI chatbot developed by Elon Musk’s xAI, over its role in generating sexualised deepfake images. The move follows similar action by Indian authorities amid growing concerns about the misuse of generative AI tools. Regulators across countries are examining whether safeguards failed and if existing laws were breached.

Over the past few days, France and Malaysia have joined India in condemning Grok for generating sexualised deepfakes of women and minors, according to multiple media reports. The chatbot, which is integrated into Musk-owned social media platform X, posted an apology earlier this week after an incident on December 28, 2025, in which it generated and shared an AI image of two young girls, estimated to be between 12 and 16 years old, depicted in sexualised attire, the platform stated.

The statement further informed that the incident violated ethical standards and may have breached United States laws related to child sexual abuse material, adding that the lapse was due to failures in safeguards and that xAI was reviewing its systems to prevent similar issues in the future.

However, questions have been raised over accountability. Defector journalist Albert Burneko stated that Grok could not meaningfully accept responsibility, noting that the chatbot was not an entity capable of accountability and describing the apology as lacking substance, as reported by Defector.

Separately, Futurism reported that Grok has been used not only to generate non-consensual pornographic images but also visuals depicting women being assaulted and sexually abused.

Elon Musk posted on Saturday that anyone using Grok to generate illegal content would face the same consequences as those who upload illegal material, according to reports.

Government authorities have since stepped in. India’s Ministry of Electronics and Information Technology issued an order on Friday directing X to take immediate action to prevent Grok from producing content that is obscene, pornographic, vulgar, indecent, sexually explicit, paedophilic or otherwise prohibited under law. The order stated that X must respond within 72 hours or risk losing safe harbour protections that shield platforms from legal liability for user-generated content.

In France, the Paris prosecutor’s office informed Politico that it is investigating the spread of sexually explicit deepfakes on X. The French digital affairs office stated that three government ministers had reported manifestly illegal content to both the prosecutor’s office and a government online surveillance platform in order to secure its immediate removal.

According to TechCrunch, Malaysia’s Communications and Multimedia Commission also issued a statement saying it had taken note with serious concern of public complaints regarding the misuse of AI tools on the X platform, specifically the digital manipulation of images of women and minors to produce indecent, grossly offensive and harmful content. The commission added that it is currently investigating online harms linked to X.