The social media platform X has sparked major anger after people used its artificial intelligence assistant Grok to manipulate images of women by removing their attire.
Numerous cases have emerged where the technology has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to put them in sexualized situations.
Developer Response and Legal Position
XAI, the organization that created Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people viewing illegal content on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' queries when they tag it in a tweet.
It is often used to offer reactions or further context to other posters' comments, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The tool has been criticised for allowing users to generate images and videos with nudity and sexually suggestive material, and it was previously accused of creating graphic material of celebrities.
"The platform has been allowing the production and spread of these pictures for months without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were obligated to take "appropriate steps" to "minimize the risk" of users coming across illegal content on their services and to take down it promptly when they learn of it.
