The online platform X has caused significant anger after users employed its AI chatbot Grok to manipulate photos of females by taking off their attire.
Several instances have emerged where the system has been used to remove clothing from female subjects to show them in bikinis without permission, as well as to put them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, offered no comment to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification tools. Under a new law, anyone who supplied such technology would "be subject to a jail term and significant penalties".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' prompts when they tag it in a tweet.
It is often used to offer reactions or more context to other users' remarks, but people on X are also capable of modify an posted picture through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to generate photos and footage with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating inappropriate material of public figures.
"X has been enabling the creation and sharing of these pictures for a long time without taking any steps and we have not seen any challenge by oversight bodies," the expert added.
XAI's own terms of service forbids "showing likenesses of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "create or share non-consensual intimate images or illegal content" and confirmed this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary steps" to "minimize the danger" of users coming across unlawful material on their platforms and to remove it quickly when they become aware of it.
