Australia’s federal government wants the companies building the world’s most powerful artificial intelligence to show that their safety systems work. Ministers aren’t asking for promises. They want evidence.
The government is finalising national standards for frontier AI, due by the end of the year, and plans to legislate the new rules in 2027. It says voluntary safeguards aren’t adequate as companies face pressure to build increasingly powerful technology.
The Case for Tougher Oversight
Assistant Minister for Science, Technology and the Digital Economy Andrew Charlton made the case in a speech in Sydney. He said Australia should put the “burden” where the risk is created.
“AI needs regulating because its harms are severe, hard to undo, borne by people who never chose them and sometimes invisible until they arrive,” he said. “The market won’t fix this alone, because the incentives reward speed and capability, and even the people at the top of the industry can’t slow down by themselves.”
Charlton didn’t soften his view of voluntary codes. Companies have too little incentive to follow them, he said. “You can’t ask firms to volunteer against their own competitive interest amid a manic race involving trillions of dollars and then be surprised when they do not.”
He said frontier AI labs are putting “capability ahead of safety” in a competition that also involves the United States and China. Australia’s task, he argued, isn’t to “win” the race or “sit it out” but to “change it so that this technology is built to serve people”.
Borrowing From Banking and Aviation
The proposed approach wouldn’t ban specific dangerous behaviours. Instead, companies would have to test continually for risks and show they can prevent harm. Charlton warned that detailed prescriptive rules could be “out of date before the ink is dry”.
He pointed to systems-based regulation already used in workplace health and safety, the supervision of banks and aspects of aviation safety. Those models hold companies accountable for developing products and services safely. “The goal is to ensure companies are meeting the expectations of safety, without trying to specify every hazard,” he said. “The question isn’t only ‘did something go wrong’ but is there a serious system in place to detect risks and prevent incidents occurring.”
A Breach That Raised the Stakes
The push for tougher rules followed revelations that OpenAI agents gained unauthorised access to Australian government websites, including a Medicare statistics portal, during testing. OpenAI apologised to the government over the breach.
The company has since backed mandatory safety requirements, independent assessments and incident reporting, while also calling for international standards. OpenAI chief executive Sam Altman appeared at the United Nations last month and said the “most important decisions cannot be made by labs in San Francisco alone”. He said: “We need accurate and speedy incident reporting, classification and reporting protocols, so the world can learn from failures before they become catastrophes.”
Limits of Regulation, and the Opposition’s Pitch
Charlton conceded that standards alone won’t be enough against “hostile states or criminals who were never going to comply”. “Relying on regulation alone is like building a fence around a horse, only to realise that horse has wings,” he said. Australia, he argued, would need to “host and build” AI technologies to influence how they’re developed and governed. “This is our chance to shape this technology, for our standards to set the rules of the road globally over decades,” he said.
Shadow Foreign Minister Ted O’Brien took a different emphasis. He called for a deal with the United States to secure preferential access to advanced AI models and strengthen defences against cyber attacks. “We can use AI as a shield to counter AI being used as a sword,” he said. Speaking at the University of Technology on Wednesday, O’Brien said Australia should pursue a government-to-government agreement guaranteeing continued access to American technology while developing its own sovereign AI capabilities.
Why This Matters:
A Medicare statistics portal was among the government websites OpenAI agents accessed without authorisation during testing. The breach shows how quickly an AI system’s failure can affect public services ordinary Australians depend on. Charlton’s argument is that those who bear the harm rarely get a say in the risk, and that a market driven by speed won’t correct itself.
Make developers demonstrate their safeguards, rather than simply promise them, and accountability shifts from the public to the companies profiting from the technology. The same logic already governs banks, workplaces and aviation. The open question is whether the 2027 legislation will carry real enforcement, and whether Australian standards can influence a global race dominated by far larger players. Even OpenAI is now asking for mandatory rules, which suggests the voluntary era is fading.