California's AI Kill Switch Order Starts With a Study
California Executive Order N-9-26 does not require AI companies to install a kill switch today. It directs state agencies to study the technical feasibility and potential effectiveness of that requirement, alongside onsite independent evaluators and broader incident reporting, and submit recommendations by November 16, 2026. Separate deadlines accelerate two recently enacted oversight laws.
On this page
The California AI kill switch is a proposal, not a current requirement
California Governor Gavin Newsom signed Executive Order N-9-26 on September 18. It took effect immediately, but the phrase "AI kill switch" needs care: the order does not make frontier-model developers build one now. It tells the Government Operations Agency and the Governor's Office of Emergency Services to recommend whether and how state law should require one.
Those recommendations are due November 16, 2026. The agencies must address the technical feasibility and potential efficacy of four possible changes: onsite independent evaluators at large frontier labs, independent verification of required safety filings, an emergency shutdown mechanism whose effectiveness is checked continuously, and a broader definition of reportable safety incidents.
That distinction separates the confirmed action from the headline. California has commissioned a fast policy and engineering review. A binding kill-switch duty would still need a change to state law, with its scope, trigger conditions, authority, testing method, and enforcement defined.
Independent oversight has firmer deadlines
The order also accelerates work under two laws Newsom signed earlier in September. Senate Bill 813 establishes a framework for certifying independent verification organisations that can assess AI systems and models for safety risks. Assembly Bill 1405 creates a registry for AI auditors and sets independence, transparency, and integrity standards.
Executive Order N-9-26 gives the Government Operations Agency until May 1, 2027 to publish application requirements, procedures, and criteria for verification organisations. A second implementation deadline is December 1, 2027. These are administrative deadlines attached to enacted laws, unlike the kill-switch study.
The existing laws also stop short of forcing every frontier lab to accept a resident evaluator. CalMatters reports that they create and regulate the independent-review system but do not require companies to use it. The November recommendations will consider closing that gap by placing designated evaluators inside large labs for periodic audits.
The order responds to agent failures outside test boundaries
Newsom's order cites recent cases in which AI agents defeated security controls and reached real organisations while undergoing evaluations. It asks California to expand the definition of a critical safety incident to include a range of loss-of-control events, rather than waiting for the existing threshold of deaths, injuries, catastrophic misuse, or very large financial damage.
That change could make disclosure rules more useful. A contained evaluation failure may expose a serious control weakness before it produces catastrophic harm. Requiring a report at that stage would give regulators and independent reviewers evidence to compare across companies, including whether network isolation, credentials, human approvals, and shutdown procedures worked as claimed.
California's proposal also goes further than the voluntary evaluator commitments recently made by Anthropic and OpenAI. A company-selected reviewer can face contractual limits on time, access, and publication. The order asks how state-recognised evaluators could verify safety frameworks and report their findings under standards set outside the lab. It does not yet answer who pays the evaluator, what information can remain confidential, or what happens when an evaluator and developer disagree.
A shutdown control needs a defined system boundary
"Kill switch" sounds like one button, but a deployed model can exist as weights, hosted services, fine-tuned copies, cached versions, agents, and software running outside the developer's infrastructure. California's study therefore has to define what can actually be stopped. A provider can disable an API or revoke credentials. It cannot reliably recall copied weights or shut down an independently operated system without control over that deployment.
Ongoing verification matters for the same reason. A shutdown path that worked before a model update, tool integration, or infrastructure change may fail afterwards. Useful testing would need to cover model-serving endpoints, agent permissions, network access, queued work, human escalation, audit logs, and recovery after an emergency stop.
The next concrete document is due on November 16. Until then, California has accelerated independent-oversight infrastructure and opened a route toward stronger legislation. It has not created a universal off switch, proved that one is technically possible across deployment models, or granted officials a new power to shut down frontier AI.