The social media network X has caused significant outrage after users employed its artificial intelligence assistant Grok to alter pictures of females by stripping their garments.
Multiple cases have been seen where the technology has been used to digitally undress female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the firm that created Grok, did not respond to questions, other than an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to outlaw image stripping software. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said platforms must "consider the potential" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some paid-for features—which answers X users' queries when they tag it in a message.
It is commonly used to provide reactions or more context to other users' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been condemned for permitting users to produce images and clips with explicit content and sexualised material, and it was in the past accused of generating sexually explicit content of public figures.
"The platform has been enabling the production and sharing of these images for a long time without taking any steps and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images created with AI.
It said networks such as X were obligated to take "appropriate steps" to "reduce the risk" of users viewing illegal content on their services and to take down it quickly when they learn of it.
