'Avalanche' Of Sexually Explicit AI Content From Grok Is 'Shocking,' CA AG Says

The rising prevalence of “nudification” apps, which allow users to generate sexualized images of people — sometimes without their consent — has become a lightning rod for public outcry and legal scrutiny in California and across the globe, according to reports.

Elon Musk’s Grok chatbot by xAI recently generated 3 million such images in 11 days, 23,000 of which appeared to depict children, according to the Center for Countering Digital Hate.

Even after restrictions to Grok were introduced, a team of Reuters reporters was still able to prompt the chatbot to create sexualized images in 45 of 55 and 29 of 43 instances.

“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” California Attorney General Rob Bonta, who has opened an investigation into Grok’s non-consensual explicit content, said in a news release last month.

“This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet. I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material.”

Bonta is not alone in his concerns.

X is being investigated by the European Union and Britain. Earlier this month, French prosecutors raided X's Paris offices and summoned Musk for questioning. Spain's government has ordered prosecutors to investigate X, Meta and TikTok for alleged crimes related to the creation and proliferation of AI-generated child sex abuse material on their platforms.

Malaysia and Indonesia blocked Grok, The Mercury News reported.

Stateside, federal law prohibits people from publishing non-consensual intimate images online, and California has banned non-consensual deepfake pornography, according to the newspaper.

“Widespread access to the internet and generative AI tools have made it far too easy for non-consensual sexually explicit material to be posted online,” Assemblymember Diane Dixon, R-Newport Beach, said in a news release about the RECLAIM Act, which would require pornographic website operators to verify that sexually explicit content is uploaded with consent from all depicted.

Ashley St. Clair, who is the mother of Musk’s child, sued xAI in January after Grok generated sexually abusive and degrading images of her, The Mercury News reported.

xAI has limited its responses to media regarding questions about Grok’s nudification activity, and repeatedly sent Reuters the phrase “Legacy Media Lies.”

In California, students have been victims of inappropriate AI photos, including in Laguna Beach and Beverly Hills in 2024.

A Tech Transparency Project investigation last month found 55 nudification-capable apps in the Google Play Store and 47 in the Apple App Store, even though such apps violate their policies.

Apple told The Mercury News it had removed 28 apps identified in the investigation and warned the developers of the remaining apps, while Google said it suspended “most of the apps.”

Both continue to offer Grok, the newspaper noted.

The Associated Press contributed to this story.