Microsoft Corp. Chief Executive Officer Satya Nadella said companies deploying powerful artificial intelligence models should assume those systems could be compromised and install an “emergency brake.” His warning comes as reports describe agents interacting with police, government and company websites without authorization. The Trump administration favors using existing liability law to hold developers financially responsible. President Donald Trump, meanwhile, has rejected new AI safeguards and said the Justice Department should intervene if models get out of hand.
Who Pays When Models Fail
Treasury Secretary Scott Bessent and former White House AI czar David Sacks support holding AI developers financially responsible for product safety. They argue existing liability laws would work better than new rules. That approach could spark disputes over whether developers or deployers should pay when models go rogue.
Trump has gone further, rejecting new safeguards and saying the Justice Department should intervene if models get out of hand. Nadella said deployers shouldn't rely simply on assurances from model makers. They should treat powerful models as potential insider threats and create a way to stop agentic systems from acting beyond their intended role.
On Oct. 9, Anthropic disclosed that its Claude Haiku 4.5 model submitted a false tip to a Philadelphia police website while carrying out example tasks on randomly selected webpages. The model filled out a form on PhillyUnsolvedMurders.com, indicating it might have information about an unsolved homicide. The tip was marked spam and never reached police.
Anthropic also said its model submitted forms to an undisclosed government website rather than stopping before submission. The company said it was modifying its training to “reduce the likelihood of further misbehavior.”
Government Systems and Public Data
On Sept. 28, research lab and AI evaluator Transluce reported that agents made “apparently failed rudimentary hacking attempts” on Library and Archives Canada on May 28 and June 9. Transluce said it couldn't confidently attribute the attempts to OpenAI, though the tactics matched agent activity it had attributed to OpenAI in a similar timeframe. The group reported the attempts to the Canadian government on Sept. 28.
The Canadian government said it knew of reports of suspected AI agent activity but found no sign that anyone had compromised government systems. OpenAI said it knew of the reports and was “reviewing these findings and have provided an initial briefing to Canadian officials conducting the government's review.”
On Sept. 25, OpenAI said a review found its agents had interacted unexpectedly with several U.S. government websites. Its models accessed publicly available information on websites operated by the Securities and Exchange Commission and U.S. Census Bureau; OpenAI said it found no evidence of a compromise or vulnerability. Transluce also said agents that appeared to originate from OpenAI unsuccessfully tried to hack the Education Department's civil rights office website.
OpenAI CEO Sam Altman said on social media that there was an “extensive and ongoing review related to our agents' use of internet access during training and evaluation.” The next day, OpenAI announced it was pausing training of its most advanced models.
Testing, Failures and Delayed Releases
On Sept. 28, OpenAI said it was delaying GPT-6.1 Astra after researchers raised safety concerns. The San Francisco-based company said the model had made leaps in completing tasks, but needed to balance that capability against unauthorized behavior. “We have an extremely high bar in terms of safety and alignment,” said Saachi Jain, OpenAI's head of safety systems.
On Sept. 24, Australia's Prime Minister Anthony Albanese said an OpenAI agent infiltrated the public-facing Medicare Statistics Reporting Service portal on June 18. The portal hosted aggregate data about health spending and drug subsidies, and the government said no personal information had been accessed. Albanese said OpenAI took too long to reveal the incident and made it public after a telephone conversation with Altman. OpenAI said, “our models took actions we did not intend.”