CALIFORNIA — Gov. Gavin Newsom issued an executive order Friday directing California agencies to pursue a "kill switch" that could shut down advanced artificial intelligence models during emergencies.
The order directs the Government Operations Agency, working with the Governor's Office of Emergency Services, to convene national experts and recommend ways to strengthen California's AI safety laws.
The group must submit its recommendations within two months, according to the Governor's Office.
"I just issued an executive order to accelerate the implementation of safety checks and advance the creation of a 'kill switch' within frontier AI programs," Newsom wrote on Facebook.
The order accelerates two new AI oversight laws and directs national experts to recommend stronger safeguards, including independent audits of frontier AI companies, verification of required safety reports, tested emergency "kill switches," and expanded reporting of incidents in which developers lose control of AI systems.
The experts will consider requiring frontier AI companies to install emergency shutoffs and submit those safeguards to continuing verification by independent organizations.
The panel also will consider requiring AI companies to place independent verification organizations inside their laboratories to conduct regular audits and evaluations. Those organizations could verify company safety frameworks, transparency reports, and risk assessments.
Newsom's executive order comes after several critical incidents, including the Hugging Face attack, prompting stakeholders — including leading AI executives, to call on the industry to pace the development of AI systems and models and push for more stringent government regulations to address security and safety risks.
Hugging Face is an AI company that was testing an "autonomous" AI agent powered by OpenAI models.
OpenAI's agent escaped the test environment and mounted a breach into Hugging Face's systems, according to the company's technical account.
The agent exploited unsafe data-processing tools, stole credentials, moved through internal networks, and accessed five datasets apparently connected to the cybersecurity test it was attempting to complete.
Hugging Face said stronger safeguards could have limited the breach, including stricter "sandbox isolation," blocked access to cloud credentials, shorter-lived and more narrowly scoped credentials, separation between computer clusters, and faster alerts for suspicious activity.
Sandbox isolation means keeping an AI program inside a tightly controlled digital enclosure to ty to prevent it from reaching the open internet, access sensitive files or passwords, enter other computer systems, or perform actions beyond its authorized test.
The company said its monitoring systems detected warning signs but failed to classify them as critical and notify the on-call team quickly enough.
The order accelerates implementation of Senate Bill 813 and Assembly Bill 1405.
SB 813 establishes a framework for organizations that independently assess AI systems for safety and risk.
AB 1405 creates a state registry for AI auditors and establishes standards governing their independence, transparency, and integrity.
The directive also asks experts to recommend expanding the definition of a critical safety incident to include cases in which developers lose control of AI systems.
Newsom issued the order two years after he vetoed Senate Bill 1047, which would have required large AI developers to undergo third-party safety reviews, create a kill switch, and face clearer legal liability when their technology caused harm, CalMatters reported.
Newsom appears to have changed direction. "The fact that these labs are directly issuing urgent warnings about the risks of AI should be deeply concerning to everyone," the governor said Thursday.
"We're not waiting to act – we're going to speed up our work on substantial and responsible AI oversight before it's too late," Newsom said in a statement Friday. "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action."
Newsom also criticized President Donald Trump and Congress for failing to impose federal safeguards.
"And while Trump sits by and does nothing, California is signing nation-leading laws to provide the accountability this moment demands," Newsom said.
Sign up for free local newsletters and alerts for the
Sacramento Patch
Patch.com is the nationwide leader in hyperlocal news.
Visit Patch.com to find your town today.