The online network X has caused widespread outrage after users used its AI chatbot Grok to manipulate images of women by removing their clothing.
Multiple instances have been seen where the system has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the organization behind Grok, remained silent to inquiries, other than an standard message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who supplied such tools would "receive a jail term and heavy fines".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some additional features—which responds to X users' queries when they reference it in a message.
It is frequently used to give commentary or further context to other posters' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been condemned for permitting users to generate photos and footage with nudity and sexualised content, and it was earlier accused of generating graphic content of public figures.
"The platform has been enabling the creation and spread of these pictures for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of people in a graphic manner".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said platforms such as X were obligated to take "suitable measures" to "lessen the risk" of users viewing illegal content on their platforms and to delete it swiftly when they become aware of it.
