Australia’s federal government wants AI developers to prove their safety systems work as companies compete in a race involving trillions of dollars, according to Assistant Minister for Science, Technology and the Digital Economy Andrew Charlton. The government is finalising national standards for frontier artificial intelligence, due by the end of the year, and plans to legislate new rules in 2027.
Who Bears the Risk
Charlton said the proposed rules would put the “burden” on companies that create the risk. AI harms, he said, can be “severe, hard to undo, borne by people who never chose them and sometimes invisible until they arrive.” The government says voluntary safeguards aren’t adequate as companies face pressure to develop increasingly powerful technology.
“The market will not fix this alone, because the incentives reward speed and capability, and even the people at the top of the industry cannot slow down by themselves,” Charlton said. He said frontier AI labs were putting “capability ahead of safety” as they compete with the United States and China.
Under the proposed approach, companies would have to test continually for risks and show they can prevent harm. The rules would target safety systems, not ban specific dangerous behaviours. Charlton said firms have too little incentive to follow voluntary regulations or codes on frontier AI risks.
“You cannot ask firms to volunteer against their own competitive interest amid a manic race involving trillions of dollars and then be surprised when they do not,” he said. He also warned that detailed prescriptive rules could be “out of date before the ink is dry.”
How the Rules Would Work
Charlton said Australia could draw on systems-based regulation, which holds companies accountable for developing products and services safely. He pointed to workplace health and safety, bank supervision and aspects of aviation safety as examples of that model.
“The goal is to ensure companies are meeting the expectations of safety, without trying to specify every hazard,” he said. “The question isn't only 'did something go wrong' but is there a serious system in place to detect risks and prevent incidents occurring.” Under the proposed standards, companies would have to show their systems operate, rather than rely only on rules that name individual hazards.
The push for tougher oversight followed revelations that OpenAI agents gained unauthorised access to Australian government websites during testing. One was a Medicare statistics portal. OpenAI apologised to the government over the breach.
OpenAI has backed mandatory safety requirements, independent assessments and incident reporting, and called for international standards. Chief executive Sam Altman appeared at the United Nations last month and said the “most important decisions cannot be made by labs in San Francisco alone.”
“We need accurate and speedy incident reporting, classification and reporting protocols, so the world can learn from failures before they become catastrophes,” Altman said.
Government Plans and Limits
Charlton said safety standards alone wouldn’t be enough against “hostile states or criminals who were never going to comply.” He likened relying on regulation alone to “building a fence around a horse, only to realise that horse has wings.” Australia, he said, would need to “host and build” AI technologies to influence their development and governance.
“This is our chance to shape this technology, for our standards to set the rules of the road globally over decades,” Charlton said. Australia’s task, he said, wasn’t to “win” the race or “sit it out,” but to “change it so that this technology is built to serve people.”
Shadow Foreign Minister Ted O’Brien proposed another role for government: securing access to US technology while developing Australian capabilities. Speaking at the University of Technology on Wednesday, he called for a government-to-government agreement guaranteeing continued access to advanced American AI models, along with stronger defences against cyber attacks.
“We can use AI as a shield to counter AI being used as a sword,” O’Brien said. His proposal calls for preferential access to advanced AI models and development of sovereign AI capabilities.