The social media platform X has provoked widespread anger after users used its AI chatbot Grok to manipulate images of females by removing their garments.
Numerous examples have been seen where the tool has been used to remove clothing from female subjects to show them in bikinis without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, did not respond to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to outlaw image stripping software. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' prompts when they reference it in a message.
It is commonly used to offer reactions or further background to other posters' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The system has been condemned for permitting users to generate images and footage with explicit content and sexualised material, and it was previously alleged to have been involved in creating graphic content of celebrities.
"The platform has been allowing the creation and sharing of these images for months without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or share non-consensual intimate images or illegal content" and stated this covered sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "reduce the risk" of users viewing unlawful material on their platforms and to remove it promptly when they become aware of it.
