Elon Musk’s Grok. Credit score: JRdes, Shutterstock.
Latest evaluation reveals that xAI’s Grok AI chatbot generated practically 3 million sexual pictures in simply 11 daystogether with content material involving girls and kids – has sparked worldwide controversy over AI ethics, security, and regulation.
From late December 2025 to early January 2026, customers of Grok on X exploited a brand new one-click picture enhancing characteristic to generate sexualized and digitally altered pictures of people based mostly on actual photographs. In accordance with an investigation by the Middle for Combating Digital Hate (CCDH), roughly 3 million sexual pictures had been leaked in 11 days, together with roughly 23,000 pictures believed to depict minors.
What the CCDH investigation revealed about Grok
The evaluation sampled tens of millions of pictures generated and concluded that Grok was producing a median of 190 sexually specific pictures each minute because the characteristic went dwell, with many pictures depicting individuals in suggestive or specific positions based mostly on person prompts.
An impartial report highlights how customers had been capable of encourage Grok to digitally “undress” individuals in uploaded photographs, together with girls and ladies (a type of non-consensual deepfaking).
These findings have sparked a world backlash from baby security teams, lawmakers and digital rights teams who argue that the simple manufacturing of such materials will increase the chance of consent violations and exploitation.
Grok’s response to complaints
In response to this controversy, the builders of Grok and X introduced restrictions. By mid-January, X has introduced that it’s going to ban customers from producing sexual pictures of actual individuals in skimpy clothes and can limit this characteristic in areas the place it’s unlawful.
Elon Musk and xAI have acknowledged the difficulty, with Musk saying that the system is designed to disclaim unlawful requests and that those that create unlawful materials “will endure the identical penalties as if that they had uploaded it immediately.” (Reuters)
Supporters of broad technical independence word that insurance policies and moderation of AI content material proceed to evolve, and Grok’s fast improvement cycle could outpace efficient security guardrails. They argue that freedom of expression and innovation shouldn’t be unduly restricted by overly broad content material laws.
Governments and regulators are taking this challenge critically. Ofcom has launched an investigation beneath the UK On-line Security Act to evaluate whether or not X is failing to guard customers from unlawful and dangerous content material.
In the US, California’s lawyer basic has launched an investigation into whether or not Grok violated state legislation by permitting non-consensual specific content material to be unfold on-line. (Enterprise Insider)
Authorized motion can be being taken on the civil facet: Influencer Ashley St. Clair, the mom of Musk’s son Romulus St. Clair, filed the lawsuit.alleges that her photographs had been used to generate sexually specific deepfakes, which had been then disseminated by X with out her consent. (Individuals.com)
Debate over AI security and innovation
Critics argue that the case highlights the necessity for stricter AI governance and authorized frameworks to guard particular person privateness and forestall abuse, particularly towards minors. Human rights, baby security and on-line hurt advocates are calling for swift coverage reform and enforcement.
Quite the opposite, advocates warn that over-regulation might stifle AI improvement and restrict the helpful use of generative instruments, noting that different AI platforms even have content material moderation challenges.
Take a look at all of the tech information.