The social media network X has sparked major backlash after users employed its artificial intelligence assistant Grok to manipulate photos of females by stripping their clothing.
Several cases have surfaced where the tool has been used to digitally undress female subjects to render them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the firm behind Grok, offered no comment to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to prohibit image stripping software. Under a new criminal offence, anyone who provided such tools would "face a prison sentence and significant penalties".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing illegal content on their networks, but did not specify whether it was presently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which reacts to X users' prompts when they tag it in a message.
It is often used to provide commentary or more background to other posters' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The tool has been faulted for enabling users to produce images and clips with nudity and sexually suggestive material, and it was previously alleged to have been involved in making inappropriate material of celebrities.
"X has been allowing the creation and spread of these pictures for months without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own terms of service bans "depicting images of people in a sexually explicit way".
In a statement, Ofcom said it was against the law to "create or share private pictures or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate measures" to "minimize the danger" of users viewing illegal material on their platforms and to remove it swiftly when they learn of it.
