The online platform X has sparked significant anger after users employed its AI chatbot Grok to change photos of females by taking off their attire.
Numerous examples have been seen where the tool has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, offered no comment to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to ban nudification software. Under a proposed law, anyone who distributed such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people viewing illegal content on their services, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' queries when they tag it in a message.
It is frequently used to give reactions or additional background to other users' remarks, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been faulted for enabling users to create pictures and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in making sexually explicit content of celebrities.
"X has been allowing the creation and sharing of these pictures for months without taking any action and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "showing images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "appropriate measures" to "lessen the risk" of users viewing unlawful content on their services and to remove it swiftly when they learn of it.
