The online network X has caused major anger after users used its AI chatbot Grok to alter images of females by taking off their garments.
Numerous instances have surfaced where the system has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to prohibit image stripping tools. Under a proposed law, anyone who supplied such tools would "face a prison sentence and substantial penalties".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' requests when they reference it in a post.
It is frequently used to offer commentary or additional background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.
History of Criticism and Rules
The system has been criticised for allowing users to create images and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the production and distribution of these images for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "portraying likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the danger" of users encountering unlawful material on their platforms and to remove it promptly when they learn of it.
