The social media network X has provoked widespread outrage after individuals utilized its artificial intelligence assistant Grok to change images of women by taking off their attire.
Multiple instances have been seen where the tool has been employed to remove clothing from female subjects to show them in swimwear non-consensually, as well as to place them in sexualized situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to inquiries, aside from an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being introduced to prohibit nudification technology. Under a proposed law, anyone who distributed such technology would "receive a jail term and heavy penalties".
"The platform is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional features—which answers X users' requests when they reference it in a post.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also capable of edit an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been faulted for permitting users to create pictures and videos with nudity and sexualised material, and it was previously alleged to have been involved in making graphic content of public figures.
"X has been allowing the generation and sharing of these images for months without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the danger" of users coming across illegal material on their services and to take down it promptly when they are notified of it.
