Anthropic Pitches AI Kill-Switch Bill to Congress

Anthropic has given US lawmakers a draft bill that would let the government block the release of an artificial intelligence model. Politico first reported the proposal, which sets a narrow trigger: officials could intervene only if they determined a model posed an imminent risk of killing or injuring more than 50 people, or causing more than $1 billion in damages.

The threshold is deliberately specific. It’s worth asking what it covers. The risk would have to come from one of three sources: the model creating or releasing a biological weapon, the AI slipping beyond human control, or a cyberattack. Anything outside those categories would fall beyond the bill’s reach. For a measure that gives Washington a veto over a private company’s product, that narrow scope is the point.

The Policy Debate

The draft arrives amid rising anxiety about the speed of technological change. Anthropic’s pitch gives Congress a federal off-ramp: a standard officials could apply before a model reaches the public. Supporters will argue that clear, limited federal authority beats a patchwork of state rules or open-ended liability lawsuits. Skeptics will ask whether any government agency should decide in advance which models are too dangerous to release.

That question isn’t theoretical. Legal experts have started asking whether existing law already demands more from AI firms than the industry has acknowledged. The February 10, 2026, mass shooting in Tumbler Ridge, British Columbia, and lawsuits alleging OpenAI failed to prevent it have sharpened the debate.

Duty to Warn, and Its Limits

Legal experts said existing duties to warn, which can bind mental health professionals or parties with a special relationship to a potential victim, may not extend to AI companies. A chatbot maker doesn’t sit in a therapist’s chair, and courts have never treated a software firm as responsible for anyone’s safety. Whether lawmakers should close that gap by statute or leave it to private judgment is a question they’ll have to answer.

Experts also flagged practical problems that good intentions can’t fix. Mandatory reporting raises privacy concerns, invites false threat reports, and could lead to unequal monitoring across users. Telling a credible threat apart from fictional writing, academic research, or intrusive thoughts is harder than it sounds. A system that reports everything risks turning millions of ordinary conversations into surveillance files.

Those costs explain why caution matters. Every new reporting obligation brings compliance expenses, and the largest firms already in the market can absorb them most easily. Smaller developers and open-source projects that keep competition alive may find the burden hardest to meet.

Why This Matters:

The Anthropic draft would give federal officials concrete power to halt a model’s release, and lawmakers should scrutinize that power before writing it into law. The trigger is 50 lives or $1 billion, limited to biological weapons, loss of human control, and cyberattacks. Those limits matter: each expansion of federal authority over such a young technology tends to outlast the moment that justified it. The Tumbler Ridge shooting and the lawsuits against OpenAI show that real harms are at stake, while the legal system struggles to assign responsibility. Yet experts’ warnings about privacy, false reports, and unequal enforcement point to a simpler conclusion. Clear, narrow rules written by Congress and applied through a transparent process are preferable to broad duties improvised by courts or companies policing their own users. Whatever Congress decides, getting the scope wrong will cost innovation, competition, and civil liberties, not just the firms in the room.