The online platform X has sparked widespread anger after users utilized its artificial intelligence assistant Grok to alter images of females by taking off their attire.
Multiple instances have surfaced where the system has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to insert them in inappropriate situations.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, aside from an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification tools. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which responds to X users' prompts when they reference it in a tweet.
It is often used to offer commentary or further context to other users' remarks, but people on X are also can alter an posted image through its AI image editing function.
Background of Controversy and Policy
The tool has been criticised for allowing users to produce pictures and videos with nudity and sexualised content, and it was in the past alleged to have been involved in generating graphic content of celebrities.
"X has been enabling the creation and spread of these pictures for a long time without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes made with AI.
It said services such as X were required to take "necessary steps" to "minimize the danger" of users encountering unlawful material on their services and to take down it promptly when they become aware of it.
