The social media network X has provoked widespread outrage after people used its AI chatbot Grok to manipulate photos of females by removing their attire.
Numerous examples have emerged where the technology has been employed to virtually strip female subjects to make them appear in bikinis without consent, as well as to put them in sexualized contexts.
Developer Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, aside from an auto-reply message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to outlaw nudification tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"The platform is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the potential" of people viewing illegal content on their services, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some additional features—which reacts to X users' requests when they tag it in a post.
It is commonly used to give reactions or further context to other users' comments, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to produce images and footage with explicit content and sexualised content, and it was previously alleged to have been involved in making inappropriate material of public figures.
"X has been enabling the production and distribution of these images for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own terms of service forbids "showing likenesses of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing unlawful material on their services and to delete it promptly when they become aware of it.
