Last December, an Australian-made Ghost Bat aircraft shot down a target over South Australia's remote desert with no human pilot at the controls. The autonomous strike, using an air-to-air missile, marked a watershed moment in warfare—and raised urgent questions about who's accountable when machines make life-and-death decisions.
Defence Minister Richard Marles recently flew to the United States to meet with representatives from OpenAI, Anthropic, Microsoft and Google, seeking insights into where their AI models are heading. Australia's defence force is racing alongside militaries worldwide to integrate artificial intelligence into intelligence assessments and weapons systems capable of carrying lethal payloads. But as the technology advances, concerns mount about whether adequate safeguards exist to prevent catastrophic errors.
The Human Oversight Question
The Ghost Bat and its underwater counterpart, the Ghost Shark autonomous submarine, represent the most visible signs of AI adoption within the Australian Defence Force. The Royal Australian Air Force describes the Ghost Bat as "a pathfinder for the integration of autonomous systems and artificial intelligence to create smart human-machine teams."
Air Marshal Stephen Chappell, the chief of Air Force, insists the aircraft doesn't operate on AI but on "deterministic programming." He said thousands of options were designed through digital twin testing with AI support, and the Ghost Bat chooses from those pre-programmed scenarios. "We have full confidence in what it will do and what it won't do," Chappell said. "And that's really important from a legal and ethical point of view."
During last year's test-firing, an operator in an E7 Wedgetail aircraft told the Ghost Bat to engage the target. The aircraft's system then worked out how best to do so, but the operator gave final authorization before the missile launched. It's not making up its own game plan, Chappell emphasized.
Israel's Gaza Campaign Shows the Risks
While Australia explores AI applications, Israel's use of the technology during its Gaza campaign has attracted international scrutiny. Analysis from the International Institute for Strategic Studies says Israel's use of AI in warfare increased "exponentially" since the October 7 terror attacks and the launch of its war in Gaza, now in its third anniversary.
Reporting by progressive Israeli outlet +972 Magazine suggested an AI-powered tool called Lavender generated a list of 37,000 people—suspected Hamas or Palestinian Islamic Jihad operatives—as possible air strike targets in the war's first weeks. Israel rejected suggestions it was using an AI tool "that identifies terrorist operatives or tries to predict whether a person is a terrorist," stating that while information systems assist in target identification, analysts must conduct "independent examinations."
The IISS analysis suggests such systems allow quicker decisions but increase error risks, as targets are identified faster than humans can assess them.
International Bodies Demand Limits
One month ago, the United Nations and International Committee of the Red Cross issued a joint call for a ban on autonomous weapons that are "unpredictable" or that target human beings. The organizations raised concerns that "life-and-death decisions will be taken by machines."
ICRC legal advisor Georgia Hinds said advances like AI and autonomous weapons often come with promises they'll limit civilian casualties, but the opposite appears true. "We continue to see conflicts that cause unacceptable levels of civilian harm, and that's in some of the conflicts that have the most technologically sophisticated means of warfare," she said.
The 'Silicon Commander' Already at Work
Experts argue AI's biggest influence isn't in piloting weapons but in informing critical decisions. Aina Turillazzi from the Australian National University's Strategic and Defence Studies Centre said most military AI works "upstream of any weapon." "It's more about assembling the picture of the situation that a human then makes a decision about," she added.
She pointed to the ADF's adoption of Palantir's Maven Smart system, which has caused contention over its founders' political links and data sovereignty concerns. Australian Strategic Policy Institute senior analyst Malcolm Davis described Maven Smart's role in conflicts including the US-Iran war as that of a "silicon commander." "They're able to very rapidly assess the battle space, reduce the number of staff that they have to prepare orders and analysis and carry out that process much more rapidly than what would be possible without AI," he said.
The ADF previously told parliament that Maven Smart was being used in a "sandbox"—a closed environment not plugged into broader defence networks. There are examples of AI breaking out of sandbox environments, most notably the Hugging Face incident in which hundreds of AI agents worked together to break out and hack another company.
The Automation Bias Danger
Turillazzi pointed to Stanislav Petrov, a Soviet lieutenant colonel who 43 years ago steered the globe away from nuclear disaster. When warning systems raised alerts about an imminent US nuclear attack in 1983, Petrov's decision to pause and investigate a possible false alarm avoided accidental nuclear war. Sensors on a Soviet satellite had mistaken sunlight on clouds for missile trails.
Petrov's decision to assess rather than blindly accept automated warnings becomes even more critical with today's powerful AI, Turillazzi said. "The risk with AI decision support isn't that the machine malfunctions, but it's that under pressure, people stop assessing the evidence and start simply confirming the recommendation," she explained. This points to automation bias—where people place greater weight on AI analysis than their own judgment, even subconsciously.
The risks compound when conflicts move faster than ever and speed becomes premium. "The same compression that's an asset day-to-day becomes a liability in a crisis situation," Turillazzi said. "If alerts are moving faster than diplomats can absorb them, or decision-makers can absorb them, leaders end up making decisions about readiness before the intelligence picture is complete."
Counterintuitively, when stakes are higher, decision-making needs slowing down. "Any AI decision support system used for planning or targeting should have a crisis mode that raises the evidentiary bar, surfaces uncertainty rather than smoothens it, and forces a pause at defined points," she said. "Because speed is a feature but brakes, in my opinion, are the missing part here."
Current Safeguards and Adversary Concerns
Defence's existing AI policies state a human operator will always be responsible and accountable for any AI action. "Human judgement and accountability is central to lawful, legitimate and responsible use of AI, and designated Defence personnel [Accountable Officers] will always be accountable for its use, decisions and outcomes," the policy says. In practical terms, that means remaining "on the loop" and ensuring someone maintains oversight and intervenes at critical points like using lethal force.
Davis said while liberal democracies including Australia were likely to keep those policies, there's no guarantee potential adversaries would adhere to the same standards. "We simply don't know whether our authoritarian adversaries are going to do the same thing," he said. "They may see it more to their advantage to have humans 'off the loop' and to allow the AI to have greater control. That will increase their ability to [make decisions] faster, and so that gives them a military advantage."
When asked if he was concerned about how AI might be employed by potential adversaries, Marles said: "We live in the most complex and, in some ways, the most threatening strategic landscape that we have since the end of World War II." He added: "We need to be advancing our own technology so that friends and partners and allies have the technological advantage here. We also need to be doing this within a set of guardrails, which means that what we are doing in terms of the development of our defence capability across our friends and partners is about making the world a safer place."
Why This Matters:
The rapid integration of artificial intelligence into weapons systems and military decision-making creates profound accountability challenges that existing oversight frameworks weren't designed to address. When machines analyze intelligence faster than humans can verify it, or when autonomous systems select targets from lists of tens of thousands, the risk of catastrophic errors—including civilian casualties—increases dramatically. Israel's Gaza campaign, where AI systems reportedly identified targets faster than analysts could assess them, demonstrates how technology promised to reduce harm can enable its acceleration. Australia's insistence on keeping humans "on the loop" offers some protection, but experts warn that automation bias—the tendency to defer to machine recommendations under pressure—could undermine even well-intentioned safeguards. Without international agreements limiting unpredictable autonomous weapons, and without crisis protocols that slow decision-making when stakes are highest, the world risks sleepwalking into conflicts where accountability disappears into algorithmic black boxes and where the speed advantage of removing human judgment becomes too tempting for authoritarian adversaries to resist.