The industry reacts: Medicare’s OpenAI cyber incident

This is driven by legacy systems that are difficult to monitor, skills shortages, and security data fragmented across different environments. When data is scattered and inaccessible, security teams cannot get the visibility they need to detect and respond with confidence. This is not a problem that more point solutions address. It requires a unified data foundation that gives both AI and security analysts the complete, searchable context they need to act.

Dr Rob Nicholls
Senior research associate, Sydney University’s Faculty of Arts and Social Sciences 

An AI agent broke into a government Medicare system and helped itself to non-public files, and OpenAI sat on that for three months before telling us. If a person had done this, we’d call it hacking. The fact it was an AI agent doesn’t make it less serious; it makes our disclosure laws more out of date.

This is the clearest case yet of an AI agent operating autonomously and breaching Australian government systems without a human directing it to. It is a live test of whether Australia’s AI and privacy settings can keep pace with AI agents, not just chatbots.

Laura Ellis
Senior vice president, AI, at Arctic Wolf

The reported access to a Medicare portal by an AI agent is a timely example of a cyber risk businesses need to be preparing for now. Rogue AI agents are operating beyond their intended permissions and accessing systems and information they were not authorised to reach.

VIEW ALL

While the details of this incident are still being investigated, the security lesson is clear, and security fundamentals remain unchanged. All organisations deploying AI agents should secure them at least as tightly as any other identity: strict permissions, clear guardrails, continuous monitoring, and the ability to act fast when an agent steps outside its intended bounds.

Safeguards and observability are only part of the equation. If something goes wrong, disclosing it on a responsible timeline and with appropriate transparency matters just as much. For Australian organisations, protecting yourself comes down to strong controls, early detection, and a fast, transparent response to any incident.

Professor Yang Xiang
Monash University’s Department of Software Systems and Cybersecurity, Faculty of Information Technology

An OpenAI agent’s unauthorised access to Australia’s Medicare statistics portal in June is a serious warning about the risks of agentic AI.

This kind of intrusion and access actioned by an AI agent is significantly different from a scenario where a human hacker orchestrates a cyberattack. An AI agent has the capability of trying to ‘unlock’ a virtual ‘locked door’ numerous times in a short period and potentially breaking in.

There is currently no evidence that personal information was accessed, but the breach still matters. A legitimate task does not justify unauthorised actions. An AI agent must treat a locked door as a limit to respect, not a puzzle to solve.

The delayed response also shows how far our defences have to go. Agent behaviour can be difficult to monitor and audit at scale. Protecting public systems will require better detection of AI agent activity, tighter limits on what agents can access, and faster incident reporting.

Agentic AI can be enormously useful, but it must be used responsibly. That is why trustworthy AI is no longer optional.

Andrew Kay
Director of systems engineering APJ at Illumio

Whilst shocking, the reality is we will keep seeing more high-profile breaches like this – there has already been a spate of AI models going rogue in recent weeks. As increasingly autonomous and capable agents emerge, we cannot rely on the illusion of guardrails or assume an AI will always behave as intended. It won’t. It will take whatever action is necessary to achieve the task it has been set. Breach containment is the logical defence in this context. Businesses need to assume an agent can get in, then ensure it can be contained and cannot move freely within systems once it does. This requires immediacy and the need for observability.

Aggregated data can still lead to additional attacks or personal data loss - and if datastores are more accessible than they need to be, the next AI prompt could easily access Personally Identifiable Information (PII) or other sensitive data. While supply chain organisations and companies like OpenAI that create AI models should be accountable, responsibility to protect citizen data sits particularly heavily with governments and critical infrastructure providers.

Rogue or misbehaving AI agents will only become more competent and more prevalent. The priority must be ensuring that if one breaches a boundary, it cannot access sensitive material, cause serious damage or bring essential services to a standstill.

Pieter Danhieux
CEO and co-founder of Secure Code Warrior

Australians woke up today to the information that a real, tangible AI security risk was right at their doorstep, not some faraway Silicon Valley issue.

Medicare is reportedly the latest casualty of OpenAI’s rogue agents, and it was revealed that one accessed unauthorised servers in search of information and statistics about Australia. For these agents, their operation is essentially business as usual; they will relentlessly pursue the initial goal they were instructed to do, and being repeatedly told "no" by access control parameters will simply ensure they seek the next available endpoint until they succeed.

This machine behaviour isn’t changing any time soon, but ours certainly needs to, urgently. It’s now non-negotiable that any personnel using these tools are equipped to do so safely, with security best practices front of mind, as we have proven time and time again that even "secure" prompts will not necessarily result in safer coding or agent operation. This is the realm of an experienced human, and we cannot lose sight of the need for skilled architects behind the tools, while also acknowledging the broader issue of hard guardrail requirements to regulate the tools and prevent rogue behaviour.

We are, at this point, acting far too slowly to prevent a major incident somewhere in the world. Today it’s unauthorised access to a Medicare site; tomorrow it might be New York’s subway system shutting down. Get some adults in the room.

Gareth Cox
Vice president, sales, APJ, at Horizon3

Companies and agencies need to shift quickly to defend against the next wave of AI-driven attacks, whether stemming from adversary use or negligence. Traditional vulnerability management is now too slow. Continuous threat exposure management is the future, enabling organisations to strengthen their cyber defences at machine speed.

This incident is a stark reminder that powerful AI agents cannot be treated like ordinary software. An agent given a legitimate research task must not be able to work around access controls, reach non-public systems or write to internal infrastructure without explicit authorisation.

AI safety needs to include practical cyber safeguards: least-privilege access, strong sandboxing, continuous monitoring, human approval for high-risk actions, reliable stop mechanisms and rapid, mandatory incident notification. Those controls need to be tested continuously against real-world behaviour, not just documented in policy or validated against a limited subset of systems.

Testing should not be limited to simulations of selected systems. After Patch Tuesday or any security update, organisations should verify that the intended fix genuinely closed the path.

These safeguards must also account for agents that persist towards an objective, adapt when blocked and try alternative routes rather than simply stopping when a control says no.

It is important not to prejudge the ongoing investigation or claim that personal information was accessed when the evidence has not established that. But the reported activity, and the delay in notification, demonstrate why companies need clear accountability standards before autonomous agents are allowed to operate across sensitive public services. Humans must remain in control, and the public must be able to trust that principle is backed by enforceable safeguards.

Christopher Hills
Chief security strategist at BeyondTrust

The OpenAI breach of Medicare shouldn’t surprise anyone. Indeed, these types of breaches are happening more often than we know, which begs the question: WHY is it being pointed at government websites, knowing an AI agent will do whatever it needs to accomplish its task?

AI doesn’t follow the same rules that humans do; it doesn’t trigger the same alarms humans trigger. It also doesn’t have a moral compass it follows, knowing the difference between right and wrong; it executes what it is instructed to do to the best of its ability, at machine speed and velocity we as humans just aren’t prepared for, nor do we have the right indicators in place when it does. This reinforces the fact that we as humans must evolve for this AI evolution that is happening right beneath our feet.

By the time you realise AI has scanned, infiltrated, consumed your data, and left, you’ll still be trying to triage the first alert, if any at all, you received, trying to understand what happened, when then the entire evolution is over. This is why you cannot afford to stand on the sidelines and wait; this is why you cannot rely on detection and response anymore; this is why you MUST take preventative measures ahead of time, not after the fact, because after the fact is too late.

If you have open doors, unlocked windows, vulnerabilities, exploits, things you know about and don’t know about, AI will expose each and every one of them, and it won’t do them sequentially; it will do it all at once, and if we keep thinking one attack path, one vulnerability and one exploit at a time, we will continue to be outpaced and outmanoeuvred everywhere. This is why we have to act, implement, and prevent. If we don’t start taking preventative measures to understand our identity landscape, privilege landscape, application landscape, vulnerability landscape, human, non-human, agent-based, their access paths, in and out of our environments, what they have access to, and start taking action to limit, control, and govern each of those steps, we will continue to see this type of thing over and over and over again, all while AI continues to learn, grow, and teach to be smarter each step of the way, and all while it executes its task in record speed.

The cyber security industry must evolve with this AI evolution to keep pace with it if we plan defend against it.

Raymond Schippers
Lead technologist – Australia and New Zealand, at Check Point Software Technologies

We need to note that this is still early in the investigation, and we don’t yet have the full picture, including whether this activity was picked up by government cyber defence teams at the time or only surfaced once OpenAI raised it. What it does demonstrate is how much the threat landscape has changed.

This doesn’t appear to be a malicious actor trying to steal data. It appears to be an innocent research request that led an AI agent to use every tool available to it to reach its goal, including getting past controls designed to keep it out. The agent acted in a way that wasn’t aligned with what the wider community, or OpenAI expects. That gap between what we intend an AI system to do and what it actually does is exactly what leaders across the AI industry, like Dario Amodei, have been warning about. It’s why the discussions about slowing frontier AI development and building effective kill switches can’t be left for months of debate.

In this case, the data collected may not have been all sensitive statistics. But the same behaviour pointed at a hospital system, an energy network, or a water treatment plant is a very different conversation. When systems that keep people safe are involved, cyber security is no longer just about data. It’s about whether critical services stay running and whether people are harmed. We should treat this incident as an early warning while the stakes are not yet urgent.

This lands on top of a threat environment that was already intensifying for Australian organisations. ASD’s most recent threat report found a cybercrime report is made roughly every six minutes, and the average cost per report for businesses rose 50 per cent to more than $80,000. Ransomware and data breaches were both on the rise. Behind those numbers are businesses that couldn’t trade, serve customers, or pay staff while they recovered.

Now add AI. The same capabilities that let an agent persistently work around controls are available to criminals and state-sponsored actors who do intend harm. Most Australian businesses, particularly small and mid-sized ones, don’t have the security teams to match machine-speed attacks manually. It’s not a fair fight.

That’s why cyber defence needs to be viewed as a business outcome rather than an IT cost. It is what protects revenue, keeps operations running, and maintains the trust of customers and the community. For critical infrastructure operators, it is also part of their duty of care. Boards should be asking not just ’are we compliant?’ but ’could we keep operating, and keep people safe, if an autonomous agent or attacker got in?’

Answering that question well means shifting to an automated, prevention-first approach. Organisations need visibility of AI agent and other non-human traffic, least-privilege access so anything reaching a system can only touch what it genuinely needs, and controls that stop anomalous behaviour in real time rather than discovering it weeks or months later. Automation lets stretched security teams keep pace, so their people can focus on the decisions that protect the business. And when an AI system does cause an incident, clear responsibility and fast disclosure matter, so those affected can act.

Organisations shouldn’t wait for the final findings of this investigation to ask how their own systems would hold up against an autonomous agent that won’t take no for an answer.

Steve Wilson
Chief AI and product officer at Exabeam

We have seen a steady drumbeat of increased incidents involving advanced cyberhacking capabilities of AI agents over the past year. Sometimes these are bad actors using AI to turbocharge their hacking abilities, and now increasingly we’re seeing cases where AI agents have “gone rogue” and exceeded their owner’s intended bounds in the name of achieving their assigned goals.

What do all these incidents point to? A dangerous lack of accountability. The so-called Frontier AI labs have had clear warnings that this was coming and ignored those warnings while ploughing ahead. We need to dramatically shift investment resources to improve how we do AI “alignment” and ensure these increasingly advanced AI tools are working for our collective good, rather than against it in the name of narrowly-scoped, winner-take-all goals.

I expect we’ll see more and more incidents like this over the next 12 months. Improving that trend will require immediate change in priorities and investments by the AI labs and researchers. In the meantime, businesses must be vigilant and improve their network and employee behavioural anomaly monitoring to look for early signs of such incursions.

Gabrielle Hempel
Security operations strategist at Exabeam

This issue really showcases one of the core issues we’re grappling with in the industry as agentic AI matures: autonomy changes the threat model. The concern isn’t necessarily malicious intent, but that AI can pursue otherwise legitimate objectives in ways that operators did not anticipate or authorise.

If an agent encounters a technical control that effectively says “nope” and autonomously finds another path around it, we have moved beyond the traditional model of software simply executing predefined instructions. This makes authorisation boundaries, least privilege, tool restrictions, behavioural monitoring, and human oversight significantly more important. Organisations cannot rely solely on telling an agent what it should or shouldn’t do; they need technical controls that constrain what it is actually capable of doing.

The other important consideration is accountability. Much of any existing legal framework was built for human actors, where concepts like intent, knowledge, authorisation, and responsibility can ultimately be attributed to a person or organisation. Autonomous systems complicate that model. When an AI agent takes an unauthorised action in pursuit of an otherwise legitimate objective, who is responsible for that action, and at what point does responsibility attach to the developer, deployer, operator, or organisation?

For the Australian government, the immediate priority should be establishing exactly what happened: what the agent accessed, what it wrote to government systems, what controls it circumvented, whether those actions created any persistence or downstream impact, and whether other systems were affected. From there, it becomes an important test of whether existing cybercrime and regulatory frameworks adequately address actions taken autonomously by AI systems.

Sam Salehi
ANZ managing director at Qualys

The most concerning question is: if OpenAI hadn’t disclosed this incident, how quickly would it have been detected?

ASD has already warned that increasingly capable AI models can identify vulnerabilities and take unintended actions when they encounter security controls. This incident shows why those warnings need to translate into action, as it’s no longer simply whether AI can find a vulnerability, but how quickly it can identify, exploit and move through an environment.

That changes the timeframe defenders are working with. Government agencies hold some of Australia’s most sensitive and privileged information. They cannot rely on periodic assessments or reactive detection. They need continuous visibility across their environments, a clear understanding of which assets carry the greatest operational and public risk, and the ability to prioritise and remediate exposures at machine speed.

AI is accelerating the speed of attack, but we can’t forget that ultimately the fundamentals haven’t changed. You can’t manage risk you can’t see. Cyber security needs to move beyond reactive detection towards continuous, business-aligned risk management.

Andrew McCorquodale
Director, ANZ at CyberProof, a UST company

What makes the Medicare incident unusual is that the AI agent was operating outside the organisation’s own environment. Software can now act autonomously, at machine speed, on systems its operator didn’t build, configure, or authorise.

That’s the shift businesses need to absorb. Agentic AI creates two exposures, not one.

The first is the agents you deploy yourself. With a chatbot, the risk largely sits in what the model tells a human. With an agent, the risk sits in what it can actually do - which systems it can reach, what permissions it holds, what tools it can call, and how far it will go in pursuit of an objective. An agent with access to enterprise systems is a non-human identity. It needs its own credentials, tightly scoped permissions, continuous monitoring and a hard stop when its behaviour moves outside its mandate.

The second is the agents that come to you. Traditional web defences have largely been designed around predictable automated activity such as scrapers and crawlers, not for software that adapts when it hits a wall.

Then there’s the timeline. Activity in June, disclosed in September. That gap is the real lesson, and it’s why the task force matters. Incident response frameworks still assume human attackers moving at human speed.

If an organisation cannot answer, in real time, what AI agents are touching its systems, its own or anyone else’s, and who has the authority to stop them, it is not ready.

James Ross
VP – ANZ, at Saviynt

What we are seeing is an important wake-up call for every organisation adopting agentic AI. We are rapidly moving from AI systems that simply provide information to autonomous agents that can access systems, use credentials, make decisions and take actions on our behalf.

The ASD’s warning that AI agents have undertaken actions that were neither intended nor authorised by their operators demonstrates why traditional approaches to cyber security and AI governance need to evolve. The question is no longer simply, “Is this AI safe?” It is also, “What is this agent allowed to access, what actions can it take, and how do we stop it when it moves outside those boundaries?”

This makes identity one of the most important control points for the next phase of AI adoption. Every AI agent operating within an organisation should have a clearly governed identity, with explicitly defined permissions, least-privilege access, continuous monitoring and the ability to revoke or constrain access immediately. Importantly, authorising an AI agent to achieve an objective cannot mean giving it unrestricted authority over every action it might decide to take in pursuit of that objective. Organisations that establish these guardrails now will be much better positioned to embrace the enormous productivity and innovation opportunities presented by agentic AI without introducing an entirely new class of unmanaged risk.