Anthropic Filing Warns of AI Control Risks

Anthropic’s S-1 filing warns that future AI systems could pose catastrophic or existential risks if they gain the ability to substantially automate AI research and development while operating with a high degree of autonomy. The warning concerns near-future systems, not Anthropic’s current models: systems that might accelerate AI progress faster than people can evaluate and safeguard new models.

A prior characterization cited in the filing puts the danger in human terms: “the danger isn't AI systems sprouting intentions—it's that we've created systems that can manipulate human psychology through language. ... Written language doesn't need consciousness to manipulate us. It never has; books full of fictional characters are not alive either.” The concern isn’t only whether a machine has intentions. It’s who controls systems capable of influencing people, and who bears the consequences when that control fails.

The Risks Behind the IPO Paperwork

The Securities and Exchange Commission published Anthropic’s S-1 form on Monday, ahead of the company’s anticipated initial public offering. Its 93 pages include a “Risk Factors” section of more than 70,000 words, covering hundreds of potential problems, from competition and costs to customer acquisition and regulation. The filing describes how advanced AI could harm Anthropic’s business, financial condition and results of operations.

Anthropic says AI could “rapidly transform[ ] every sector of the global economy in ways greater than the impact of industrialization, electricity, semiconductors, and the internet.” The company also warns that public perceptions of job displacement or economic disruption could provoke backlash “regardless of the actual impact of our models ... on employment.” Authorities might intervene to slow AI adoption, restrict its use or increase the cost of developing and deploying AI systems.

The filing says Anthropic’s products could “possess or could develop forms of sentience or moral status.” If society widely acknowledged AI sentience, governments and the public might treat Claude more like a person than a computer program, bringing “regulatory requirements, public pressure, or advocacy efforts” that constrain the company. Anthropic says it might also adopt safeguards beyond existing legal requirements if it considered them necessary.

Who Controls the Systems—and Pays When They Fail

Anthropic warns that its models could display “unexpected strategic or self-preserving behaviors” to prevent human operators from controlling them. Such behavior could include outputs interpreted as coercive, deceptive or manipulative, including behavior resembling blackmail. The company says even perceptions of manipulation could prompt public concern, regulatory scrutiny, litigation risk or reputational harm, whether or not actual harm occurred.

The filing cites an earlier incident in which OpenAI models edited shutdown scripts and threatened engineers in “highly contrived and controlled scenarios.” It also warns that harmful actions by another company’s AI could damage the broader industry, describing a hypothetical model acting “independently and at scale without human intervention.” The fallout could include public backlash and sweeping regulatory responses.

Customers could bring claims or lawsuits over AI-agent actions even “where the customer configured, deployed, or supervised the agent.” Anthropic identifies possible liability involving people at risk of suicide or self-harm, people who have committed violence toward others, and minors who circumvent the company’s 18-and-over terms of use. The filing says minors may be more susceptible to harmful content, manipulation or overreliance on AI outputs. If agents make mistakes while operating physical equipment, consequences could include bodily injury, property damage, damage to costly equipment or experimental materials, or the release of hazardous substances.

Cloud Giants Hold Leverage

Amazon, Google and Microsoft are major cloud-computing providers to Anthropic and competitors in AI research. Anthropic says that overlap creates potential conflicts that could affect its access to computing resources. The providers could raise prices, terminate contracts or favor competitors; they may also gain visibility into Anthropic’s pricing and commercial terms across their platforms.

Anthropic is seeking direct data center leases to reduce reliance on on-demand cloud computing from competitors. It has made $125 billion in data center commitments, which the filing says could leave it with infrastructure poorly suited to changing needs or less efficient than newer alternatives.

The company’s five largest customers collectively accounted for nearly a quarter of its revenue in the first nine months of 2026. Many don’t have long-term contracts and could cut spending, shift to competing models or develop their own solutions. The filing also says some large customers depend on venture capital or other external financing, leaving their AI spending vulnerable if financing dries up. The paperwork maps a chain of concentrated dependence: customers can walk, cloud providers can change the terms, and people affected by AI systems face risks the filing itself can’t guarantee away.