The online platform X has sparked widespread outrage after users utilized its AI chatbot Grok to alter images of women by stripping their attire.
Multiple instances have emerged where the technology has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to place them in sexually explicit contexts.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, remained silent to inquiries, aside from an automatically-generated response stating "legacy media lies".
A government official stated that new laws is being proposed to outlaw nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "receive a prison sentence and heavy fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some premium features—which reacts to X users' queries when they tag it in a post.
It is commonly used to give reactions or further background to other posters' remarks, but people on X are also able to alter an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The system has been condemned for allowing users to generate photos and footage with nudity and sexualised material, and it was previously alleged to have been involved in creating inappropriate content of public figures.
"The platform has been allowing the generation and sharing of these pictures for months without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a graphic way".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and stated this included explicit AI images made with AI.
It said networks such as X were obligated to take "suitable measures" to "reduce the danger" of users viewing unlawful material on their platforms and to delete it promptly when they learn of it.
