Australia rolls out new AI regulator, frontier model security obligations

The Australian government announced plans to introduce new security obligations for frontier AI developers and a new AI regulator to oversee implementation and enforcement. The safety uptick is the government's response to a June cybersecurity incident involving an OpenAI agent's unauthorized access to Australia's Medicare Statistics Reporting Service.

The proposed measures would require AI developers to implement additional cybersecurity safeguards, conduct risk assessments and report incidents involving their systems to government officials. Security controls across development and deployment stages, third-party model evaluation and ongoing safety and security framework disclosures are among the specific requirements.

The government's AI in Australia's Interests initiative will also introduce an AI safety regulator, which would work in coordination with Australia's AI Safety Institute to provide technical oversight in addition to compliance monitoring. According to the government, the regulator would have the authority to impose "significant" noncompliance penalties.

Under the proposed obligations, companies must bolster security controls, introduce independent model evaluations, release safety frameworks to promote transparency and risk mitigation efforts and provide timely incident notification to the AI regulator.

"AI offers Australia and the world generational opportunities if we seize and shape them properly, and that is exactly what we're doing here," Australian Prime Minister Anthony Albanese said during a 10 Nov. press conference. "We want to take advantage while mitigating the risks, which are there. We have the opportunity to position Australia as a global leader in this space, and to influence how the world tackles this challenge, but also this opportunity."

Deputy Prime Minister Richard Marles noted efforts to establish independent AI system assessments are a crucial aspect of strengthening AI security while also allowing innovation to expand.

He added the proposed obligations also aim to increase collaboration between Australia's regulators, including the Australian Signals Directorate, Cyber Security Center and the National Office of Cyber Security.

"This is not a technology that we can run away from. If we want to maximize our agency, if we want to maximize the benefit for Australia … we must engage, and that is at the heart of those recommendations," Marles said.

Lessons learned from the Medicare breach

Australian officials also unveiled a report detailing recommendations stemming from its investigation of the OpenAI Medicare breach. The review was launched in September after OpenAI informed the government of the incident months after it occurred.

Marles previously said while the impact on systems was minor, the delay in reporting as well as an AI agent's unauthorized access to government systems is "completely unacceptable."

The review outlined 18 recommendations aimed at strengthening Australia's ability to prevent and respond to AI-related cybersecurity incidents. Among the suggestions was mandatory reporting requirements for AI developers in the case of unauthorized access or system breaches.

During the press conference, Home Affairs and Cyber Security Minister Tony Burke said as the digital landscape continues to grow, "the mandatory reporting when artificial intelligence goes rogue gives us a chance to be able to make sure that when this happens, the reporting occurs and occurs immediately."

Burke added Australia's laws need to "keep pace with the technology" while ensuring the government and organizations can still leverage AI "to find vulnerabilities before others do."