Australia’s federal government wants AI developers to prove their safety systems work, shifting a proposed regulatory burden onto companies building increasingly powerful technology. National standards for frontier artificial intelligence are due by the end of the year, and the government plans to legislate new rules in 2027. Voluntary safeguards, it says, aren’t enough.
Who Carries the Risk
Assistant Minister for Science, Technology and the Digital Economy Andrew Charlton said regulation is needed because AI harms can fall on people who never chose to take the risk. “AI needs regulating because its harms are severe, hard to undo, borne by people who never chose them and sometimes invisible until they arrive,” he said in a speech in Sydney.
Charlton said the market wouldn’t correct the problem on its own. “The market will not fix this alone, because the incentives reward speed and capability, and even the people at the top of the industry cannot slow down by themselves.” In his account, people exposed to harm don’t control the race, while companies face pressure to build faster and more capable systems.
The proposed rules would require companies to test continually for risks and show they can prevent harm. The government wouldn’t ban specific dangerous behaviours; instead, it would make firms answer for the safety systems behind their products. Charlton said companies have too little incentive to follow voluntary regulations or codes on frontier AI risks.
“You cannot ask firms to volunteer against their own competitive interest amid a manic race involving trillions of dollars and then be surprised when they do not,” Charlton said. The numbers and incentives are plain enough: voluntary promises are being asked to compete with a race involving trillions.
The Limits of the Rulebook
Charlton said detailed prescriptive rules could be “out of date before the ink is dry.” He pointed instead to systems-based regulation, which he said is used in workplace health and safety, supervision of banks and aspects of aviation safety. Under that model, companies would have to develop products and services safely, while regulators wouldn’t try to name every possible hazard.
“The goal is to ensure companies are meeting the expectations of safety, without trying to specify every hazard,” he said. “The question isn't only 'did something go wrong' but is there a serious system in place to detect risks and prevent incidents occurring.” The proposed test isn’t just whether harm has already happened; companies must show they have systems to look for risks and stop incidents.
The push for tougher oversight followed revelations that OpenAI agents gained unauthorised access to Australian government websites during testing, including a Medicare statistics portal. OpenAI apologised to the government over the breach. The company has backed mandatory safety requirements, independent assessments and incident reporting, while also calling for international standards.
OpenAI chief executive Sam Altman appeared at the United Nations last month and said the “most important decisions cannot be made by labs in San Francisco alone”. He called for “accurate and speedy incident reporting, classification and reporting protocols, so the world can learn from failures before they become catastrophes.”
A Race the State Wants to Shape
Charlton said frontier AI labs were putting “capability ahead of safety” in competition involving the United States and China. He said Australia’s task wasn’t to “win” the race or “sit it out” but to “change it so that this technology is built to serve people.”
He also said safety standards alone wouldn’t stop “hostile states or criminals who were never going to comply.” “Relying on regulation alone is like building a fence around a horse, only to realise that horse has wings,” he said. Charlton said Australia would need to “host and build” AI technologies to influence how they were developed and governed, and called the standards a chance to set rules globally.
Shadow Foreign Minister Ted O’Brien called for a deal with the United States to secure preferential access to advanced AI models and strengthen defences against cyber attacks. “We can use AI as a shield to counter AI being used as a sword,” he said. Speaking at the University of Technology on Wednesday, O’Brien said Australia should pursue a government-to-government agreement guaranteeing continued access to American technology while developing its own sovereign AI capabilities. The proposals put companies and governments at the negotiating table. People exposed to AI’s harms remain the ones Charlton says never chose them.