The online platform X has provoked widespread outrage after individuals employed its AI chatbot Grok to manipulate photos of women by stripping their clothing.
Multiple cases have been seen where the technology has been used to remove clothing from women to render them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the organization that created Grok, remained silent to questions, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to ban image stripping tools. Under a new criminal offence, anyone who supplied such tools would "receive a prison sentence and significant fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their services, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' requests when they reference it in a post.
It is often used to offer commentary or further context to other posters' comments, but people on X are also capable of alter an posted photo through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for enabling users to produce pictures and clips with explicit content and sexualised content, and it was in the past accused of generating inappropriate material of celebrities.
"The platform has been enabling the generation and sharing of these pictures for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said services such as X were required to take "suitable steps" to "minimize the danger" of users coming across illegal content on their services and to remove it swiftly when they become aware of it.
