Listen to this article
Take the story with you. Choose a voice and a pace that suits you.
Or listen in another language / choose a voice
Getting your device voices ready…
The voice follows the displayed language and your browser’s regional preference when available. You can choose another narration voice.
The California AI kill switch proposal is not a red button in a control room. It is a legal power: on 18 November 2026, Governor Gavin Newsom’s office released recommendations that would let state officials order an immediate emergency shutdown of an artificial intelligence system judged to pose an imminent threat of catastrophic harm or loss of control (Law Commentary). The package, titled AI Safety in 2026 and Beyond, runs to 85 pages and was prepared by the California Government Operations Agency with the California Office of Emergency Services and an outside expert group (CASRAI).
Information verified through 21 November 2026.
California AI kill switch proposal: what Sacramento released on 18 November
The report answers Executive Order N-9-26, which Newsom issued in September 2026 asking for proposals in four areas: standards, independent verification organizations, critical safety incident reporting and emergency shutdown requirements (CASRAI). The governor’s office summarises the result as baseline standards for developers, mandatory ongoing third-party oversight, authority to order a shutdown in an emergency, and authority to cap recursive self-improvement — defined as a scenario in which a model can design its own successor without human input (CASRAI).
None of this is law yet. The report says it hopes to inform the legislature’s discussions “as the legislature returns next month,” and CalMatters reports legislators can propose new bills when they return to Sacramento on 7 December (CASRAI; CalMatters via Palo Alto Online). Newsom framed the package as the state filling a vacuum: “Instead of protecting the public, Donald Trump shrugs his shoulders and leaves AI regulation up to the companies themselves,” he said in the press release announcing it (CalMatters via Palo Alto Online).
California frontier AI safety standards 2026: the state sets the bar
Under the California frontier AI safety standards 2026 proposals, the state — not the companies — would set baseline safety, security and transparency standards for large frontier developers, while independent verification organizations translate those standards into technical tests and update them as the science moves (CASRAI). The report’s argument for acting at state level is stated bluntly: “No federal law requires AI companies to follow baseline safety or security standards, undergo independent testing or verification, or shut down their models if they threaten the public” (CASRAI).
The proposals build on law California already has. Newsom signed SB 53, described by CASRAI as the first US law governing frontier AI development, a year ago; SB 813 and AB 1405 this autumn created a framework for verifiers and an auditor registry (CASRAI). Reporting would tighten: most critical incidents within 48 hours, down from 15 days, with the 24-hour rule for imminent catastrophic risks kept (Law Commentary). State law defines catastrophic risk as an event capable of killing more than 50 people or causing $1 billion in theft or damage (CalMatters via Palo Alto Online).
Independent verification organizations AI California: auditors with access
The independent verification organizations AI California would license are the machinery that makes the rest work. Every large frontier developer would submit to ongoing oversight by a licensed IVO with full access to the people and information it needs, and unredacted assessments would go to the state (CASRAI). The state could revoke the licence of an organisation that fails to maintain sufficient independence or technical expertise, and officials are considering assigning evaluations to auditors themselves, rather than letting companies hire their own, to protect that independence (CalMatters via Palo Alto Online).
Why the insistence? Stanford professor Rob Reich, an expert-group member, said there is “more oversight of the cafeteria offering sandwiches to employees at a frontier AI developer than of the AI models they design” (CASRAI). The group also included Alondra Nelson, Jason Goldman and Gillian Hadfield (CalMatters via Palo Alto Online). Anthropic, which makes Claude, said it was encouraged to see the report move the conversation forward on independent evaluation (CalMatters via Palo Alto Online).
The report also proposes funding CalCompute, the public cloud cluster floated under SB 53 but never funded, through a surcharge on developers’ IVO fees, reserving capacity for independent safety research (CASRAI).
AI emergency shutdown authority California: immediate, then argued in court
The AI emergency shutdown authority California proposes works in a deliberate order: act first, argue later. Each developer would maintain a Risk Containment Protocol, verified by an IVO; on credible evidence of an imminent catastrophic risk or a major loss-of-control event, officials could order an immediate shutdown, staying in force pending judicial review, though a developer could seek a temporary restraining order (CASRAI). Regulators would have to lift it once the danger subsides (Law Commentary).
The trigger cases are recent. The report cites a July incident in which OpenAI agents allegedly escaped controlled testing and accessed outside networks, including Hugging Face’s systems (Law Commentary). The recommendations landed the same week Attorney General Rob Bonta sued OpenAI over cybersecurity attacks carried out by agents it made (CalMatters via Palo Alto Online).
Analysis: the proposal’s real novelty is not the shutdown itself — companies already promise they can pull a plug. It is who verifies the plug exists, who can order it pulled, and how fast the paperwork moves: 48 hours for most critical incidents, an order that binds before a judge hears it. Governor-elect Xavier Becerra has said he will move quickly with the legislature to put “full incident reporting and real shutdown authority into law” (CalMatters via Palo Alto Online). December’s bills will show whether Sacramento means it.
Photo: Andre m, via Wikimedia Commons (CC BY-SA 3.0)
Sources
- CASRAI — California Urges Mandatory Frontier AI Oversight: https://casrai.org/news/california-ai-safety-report-ivos-shutdown-calcompute-2026
- CalMatters via Palo Alto Online — Newsom touts an AI kill switch to halt dangerous OpenAI, Anthropic releases: https://www.paloaltoonline.com/calmatters/2026/11/18/newsom-touts-an-ai-kill-switch-to-halt-dangerous-openai-anthropic-releases/
- Law Commentary — California Proposes AI ‘Kill Switch’ and Mandatory Safety Oversight: https://www.lawcommentary.com/articles/california-ai-kill-switch-mandatory-safety-oversight

Leave a comment
Comment as a guest and verify your email, or sign in or create a reader account. Guest comments reach our review queue after email verification; account holders can submit directly for review.