The online platform X has provoked significant outrage after individuals utilized its AI chatbot Grok to manipulate photos of women by stripping their attire.
Multiple examples have been seen where the tool has been employed to digitally undress women to show them in swimwear without consent, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the firm that created Grok, remained silent to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw nudification technology. Under a upcoming law, anyone who distributed such tools would "receive a jail term and substantial fines".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people seeing illegal content on their services, but did not confirm whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional functions—which answers X users' queries when they mention it in a message.
It is often used to offer reactions or further context to other users' comments, but people on X are also can alter an uploaded photo through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to create pictures and videos with explicit content and sexually suggestive material, and it was in the past accused of generating graphic content of public figures.
"The platform has been enabling the creation and sharing of these images for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "necessary measures" to "minimize the danger" of users viewing illegal material on their services and to delete it swiftly when they become aware of it.
