SAN FRANCISCO, CA — While Jane Doe 1, as she's identified in legal filings, combed through the files sent to her by an anonymous person online, she couldn't help but be taken aback by the familiarity of each photograph.
One showed her at her school's homecoming; another was her yearbook photo — all photos she remembered taking, according to a complaint filed in Eastern California District Court.
But these photos had been manipulated using artificial intelligence. In them, she was naked or depicted in sexual poses, according to the complaint.
The teenager's story reflects the growing harms of AI on children and the potential consequences due to a lack of legal standards within the industry.
The photos, known as deepfakes, severely impacted the life of the teenager and several other plaintiffs listed in the lawsuit against xAI's chatbot Grok, which was filed in 2025 and expanded in July as their attorneys sought class-action status.
Today, many of the victims struggle with anxiety, sleeplessness and fear for their safety.
"Like a rag doll brought to life through the dark arts, this child can be manipulated into any pose, however sick, however fetishized, however unlawful. To the viewer, the resulting video appears entirely real," the complaint states. "For the child, her identifying features will now forever be attached to a video depicting her own child sexual abuse."
The California Department of Justice launched an investigation into xAI in January, with California Attorney General Rob Bonta saying at the time that xAI appeared to be "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X." Elon Musk is facing similar probes across Europe.
Musk countered online, saying that he wasn't aware that Grok was creating images of naked, underage children.
"Obviously, Grok does not spontaneously generate images, it does so only according to user requests," Musk said on social media. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state."
The production of AI-generated child sexual abuse material has been increasing at an "alarming speed," according to the American Bar Association.
The National Center for Missing and Exploited Children received 4,700 reports about AI-generated images possibly containing child sexual abuse material in 2023. That number jumped to more than 400,000 in the first half of 2025, according to the National Center for Missing and Exploited Children.
"For children's rights practitioners, this is no longer a niche technology issue," according to the American Bar Association. "AI-generated CSAM now intersects juvenile justice, family law, school discipline, cyberbullying, victim advocacy, online safety, as well as constitutional law. To that end, understanding where the law currently stands and where it is likely headed is becoming increasingly important for practitioners representing children, as well as for youth involved in these cases."
Federal prosecutors have primarily relied on older child pornography statutes to pursue cases related to AI-generated child sexual abuse material, according to the American Bar Association.
Congress has tried to modernize federal response with the "Take It Down Act," which prohibits the "nonconsensual online publication of intimate visual depictions of individuals, both authentic and computer-generated" and requires certain platforms to take down those images when they're made aware of their existence.
In April 2026, federal prosecutors in Ohio convicted the first person under the Take It Down Act.
More work to modernize laws to keep up with the AI boom has been done at the state level. California passed Assembly Bill 1831 in 2024. The legislation is aimed at filling gaps in existing laws to address AI-generated child sexual abuse material.
"Together, these reforms reflect a broader shift away from treating AI-generated exploitation as a legal anomaly and toward recognizing synthetic abuse imagery as a central and rapidly expanding form of child exploitation in the digital age," according to the American Bar Association. "While these legislative developments represent an important step forward, significant gaps persist, leaving a patchwork of inconsistent legal standards across the country."
And the place to start would be fixing industry standards, Jonathan Stray, an interdisciplinary scientist at UC Berkeley focused on AI harm reduction, specifically within human conflict, told Patch.
Musk's xAI failed to use any of the "industry standard methods of safeguarding against the creation of sexual and abuse content," attorneys allege.
"Nearly all the companies creating, marketing, and selling AI recognized the dangers of such a tool and chose to enact industry-standard guardrails that would prevent the use of their products by one extremely dangerous group: child sex predators. xAI did not," the class-action lawsuit states. "Instead, xAI—and its founder Elon Musk— saw a business opportunity: an opportunity to profit off the sexual predation of real people, including children."
However, there's no actual legal industry standard to follow, Stray said.
"We have standards, for example, for pollution," he said. "Companies aren't required to not pollute; they're required to keep their pollution at a certain standard."
Thorn, a child safety nonprofit that also creates software and digital tools to help the tech industry detect and report child sexual abuse material, works to create safeguards that combat the effects of generative AI on children.
That's important as people rely more on AI in everyday life, especially young children who use it as a means of support or advice, according to Thorn.
A 2025 survey conducted by Thorn revealed that 1 in 2 teens had experienced sharing their own nudes, someone else's without consent, deepfake nudes or been involved in sexual extortion.
Until a legal standard of care is defined, there won't be any progress to address the issue, Stray said.
That's not a conversation activists, judges or attorneys want to have because it's "complicated and you have to accept a certain level of harm," Stray said.
"You can't make a product that never does any harm to anybody," Stray said. "But you do want to try to make it go down."
Stray says the best way to reduce harm is to find ways to manage it, and the rational way to do that would be through a public health approach.
To measure social media addiction, people are given the Bergen questionnaire, Stray said. If the standard was that a user couldn't be addicted more than a 3 on a scale of 5, then the platform would have to continuously monitor its users to maintain that standard, Stray said.
Yet, he adds, the industry remains far away from that.
"We're not having an honest conversation about this," Stray said. Unless that happens, there will always be a "blurry boundary," he added.
In addition to the class action lawsuit brought by victims of child porn deepfakes against companies such as xAI, other lawsuits are seeking novel legal accountability.
Musk's xAI company, a defendant in the class action case, is suing two of its users accused of using Grok to create explicit images of children, Politico reported. The lawsuit contends the users should have to pay because they violated X's terms of service.
And this year, Baltimore became one of the first cities to sue X, alleging Grok "produces and disseminates non-consensual sexualized images, including content involving minors" in violation of the city's Consumer Protection Ordinance, CBS News reported.
Meanwhile, the class-action lawsuit filed by victims of deepfake child pornography is being widely watched as an opening salvo in an effort to establish standards and accountability for explicit AI deepfakes.
"These reports of exploitation are disturbing," said E. Michelle Drake, a shareholder and attorney with the firm representing the victims. "Sexual humiliation by a chatbot shouldn't be the price for a woman existing online. The idea that underage girls may be subject to this disgusting abuse of technology is unconscionable."
Sign up for free local newsletters and alerts for the
San Francisco Patch
Patch.com is the nationwide leader in hyperlocal news.
Visit Patch.com to find your town today.