Govt likely to take legal action against Grok over objectionable AI-Generated images

The Union government is considering legal action against Elon Musk-owned artificial intelligence chatbot Grok, operated through social media platform X, for alleged violations of Indian laws governing the creation and dissemination of sexually explicit and objectionable images involving women and children.

According to reports, officials in the Ministry of Electronics and Information Technology (MeitY) said the matter remains under active examination, even after Grok recently limited its image-generation feature to paying and premium users. The ministry has indicated that such a restriction will not be sufficient unless X formally declares this limitation as an official policy applicable in India.

Legal proceedings against X and its AI tool may continue while MeitY reviews the action-taken report submitted by the platform in response to a government notice.

On Wednesday, X submitted its response to MeitY’s notice, which sought clarity on how Grok allowed users to generate sexually explicit and objectionable images of women and children using simple text prompts.

In a letter issued last week, MeitY’s cyber law division directed X to “remove or disable access, without delay, to all content already generated or disseminated in violation of applicable laws”, in strict compliance with the Information Technology Rules, 2021. The ministry also instructed the platform to ensure that evidence related to such violations is preserved and not tampered with.

The ministry further emphasised that X must strictly enforce its terms of service and AI usage restrictions, and adopt “strong deterrent measures”. These include suspending or terminating accounts found to be using Grok to create sexually explicit images involving women and minors.

A day after the government’s letter, Elon Musk issued a warning on X, stating that “anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content”.

However, officials said verbal assurances or platform-level warnings would not replace compliance with Indian law, particularly when it comes to safeguarding women and children from sexual exploitation enabled by emerging technologies.

Following mounting criticism, xAI, Musk’s AI company, has restricted Grok’s image-generation tool on X to paid subscribers, according to a Bloomberg report published on Friday. The feature was initially rolled out for free, allowing widespread access.

The move comes after reports that users in India and other countries were using Grok to generate undressed and sexually explicit images of women and children. Authorities in multiple jurisdictions have since taken cognisance of the issue and sought explanations from the platform.

Despite the restriction on X, Bloomberg reported that the standalone Grok app, which operates independently of the social network, continues to allow users to generate images without a subscription, raising further regulatory concerns.

The controversy has triggered action beyond India. The United Kingdom’s Internet Watch Foundation reported that allegedly “criminal” images created using Grok were found circulating on the dark web. UK Prime Minister Keir Starmer described the images as “disgraceful” and said media regulator Ofcom has the government’s full backing to take enforcement action.

In Europe, the European Union has termed the images “illegal” and directed X to preserve internal documents related to Grok until the end of the year, as part of an ongoing regulatory review.

As governments worldwide tighten scrutiny of generative AI tools, India’s stance is expected to play a key role in shaping how global platforms deploy AI features within the country’s legal framework.