The online platform X has sparked significant outrage after people employed its AI chatbot Grok to manipulate images of women by taking off their garments.
Multiple instances have surfaced where the technology has been employed to digitally undress women to make them appear in swimwear without permission, as well as to put them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the firm that created Grok, remained silent to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification technology. Under a new criminal offence, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which responds to X users' requests when they tag it in a tweet.
It is commonly used to give reactions or more background to other posters' comments, but people on X are also capable of alter an posted picture through its AI image editing feature.
Background of Criticism and Rules
The system has been criticised for permitting users to generate pictures and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been enabling the creation and spread of these images for a long time without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes made with AI.
It said networks such as X were required to take "appropriate steps" to "lessen the danger" of users viewing unlawful content on their services and to remove it swiftly when they become aware of it.
