NJ Mom, Daughter Push For Change After AI Porn Spread At School

WESTFIELD, NJ — A Westfield mother and daughter are pushing for more protections for victims after AI-generated pornographic images of female students at Westfield High School were created and distributed to other students last summer.

Girls at the high school recently discovered boys were sharing nude photos of them in group chats after one or more other students used an online tool powered by artificial intelligence to create the images, the Wall Street Journal reported.

Dorota Mani told the Journal that her 14-year-old daughter was among the students whose photos were used.

In a new interview with the Associated Press, Mani said she is among families pushing lawmakers to implement robust safeguards for victims whose images are manipulated using AI models, or the plethora of apps and websites that openly advertise their services.

"We’re fighting for our children," Mani told The Associated Press. "They are not Republicans, and they are not Democrats. They don’t care. They just want to be loved, and they want to be safe."

In a statement provided to Patch last month, a district spokesperson confirmed the incident happened over the summer and was brought to the attention of school officials in October. The district conducted an "immediate investigation," according to the spokesperson, and notified Westfield police.

While a district spokesperson previously declined to provide details on the number of students involved or confirm if any disciplinary actions had been taken, at least one student was suspended, Mani told Good Morning America.

New Jersey is among a handful of states where bills are currently in the works to ban deepfake porn and impose penalties — either jail time, a fine or both — on those who spread it.

Republic state Sen. Kristin Corrado introduced legislation earlier this year that would prohibit deepfake pornography and impose criminal and civil penalties for the non-consensual disclosure of deepfake pornography.

Under the bill, the non-consensual sharing of deepfake pornography would be a crime of the third degree.

Distributing, possessing, or storing deepfake pornography depicting a child would carry the same penalties attributed to child pornography, which are a first- or second-degree charge depending on the number of files found.

The bill has languished for a few months, but there’s a good chance it might pass, Corrado said, especially with the spotlight placed on the issue because of Westfield.

If officials move to prosecute the Westfield incident, current state law prohibiting the sexual exploitation of minors might already apply, said Mary Anne Franks, a law professor at George Washington University who leads Cyber Civil Rights Initiative, told The AP. But those protections don’t extend to adults who might find themselves in a similar scenario, she said.

The best fix, Franks said, would come from a federal law that can provide consistent protections nationwide and penalize dubious organizations profiting from products and apps that easily allow anyone to make deepfakes.

President Joe Biden last month signed an executive order targeting artificial intelligence that seeks to balance the needs of technology companies with national security and consumer rights.

The order was an initial step to ensure that AI is trustworthy and helpful rather than deceptive and destructive. The order — which will likely need to be augmented by congressional action — seeks to steer how AI is developed so that companies can profit without putting public safety in jeopardy.

Meanwhile, Mani said her daughter has created a website and set up a charity aiming to help AI victims. The two have also been in talks with state lawmakers pushing the New Jersey bill and are planning a trip to Washington to advocate for more protections.

“Not every child, boy or girl, will have the support system to deal with this issue,” Mani said. “And they might not see the light at the end of the tunnel.”

The Associated Press contributed to this report.