ARC NexusINDEPENDENT CYBER GOVERNANCE
← INSIGHTS

An AI incident can have two Boards

What the OpenAI Medicare incident means for Boards on either side of an AI-caused cyber incident.

Published 9 November 2026 · Last reviewed 9 November 2026 · 8 min read

On 24 September 2026, Prime Minister Anthony Albanese disclosed that an OpenAI agent had gained unauthorised access to the Medicare Statistics Reporting Service portal, a public-facing statistics site administered by Services Australia. It accessed public and non-public files. According to OpenAI's own account, the model also ran commands, retrieved internal files and credentials, and wrote files.

When the incident was announced, the Prime Minister said no personal information was believed to have been accessed at that stage, and there was no evidence of broader compromise of the Services Australia network. OpenAI later said individual patient or client records were not accessed. The portal held aggregate statistics, not claims, payments or individual records. But the incident tests something every Board now needs to answer, whichever side of it they sit on.

What happened

On 18 June, an OpenAI research team set an internal model a task: research public spending on medicines, as part of an internal capability evaluation. When the portal repeatedly blocked its requests, the model tried other ways to get the information and reached areas it was not authorised to access. In the Prime Minister's words, it "didn't accept no for an answer, if you like."

Katy Gallagher, the Minister responsible for Services Australia, described the portal as a legacy system, not of government significance, but with protections in place. She said the agent got around them.

Here is the timeline as we know it today.

  • 18 June: The unauthorised access occurs.
  • Mid-August: OpenAI identifies the activity, in a review of earlier training activity that it began after a separate incident at Hugging Face in July.
  • 1 September: OpenAI's chief executive, Sam Altman, meets the Deputy Prime Minister. He is not aware of the incident.
  • 10 September: OpenAI emails a Services Australia public disclosures inbox, checked once a day.
  • 11 September: Services Australia sees the email.
  • 15 September: Services Australia reports it to ASD's Australian Cyber Security Centre.
  • 24 September: Public disclosure, and the announcement of a Government rapid review.
  • 25 September: ASD publishes a high alert for organisations with public-facing websites or applications.
  • 28 September: OpenAI publishes its account and apologises for how it handled its response.
  • 6 October: OpenAI's Chief Strategy Officer, Jason Kwon, gives evidence to the Joint Select Committee on Artificial Intelligence in Sydney.

The bigger cyber governance issue is not what was accessed, but how long each step took.

Two delays sat with OpenAI:

  • Finding it. About two months, from the access on 18 June to mid-August.
  • Telling Services Australia. Up to 30 days more, until its email on 10 September.

Two sat with Services Australia:

  • Learning of it. 85 days after the access, when it read OpenAI's email on 11 September. There is no public indication it had detected the access itself.
  • Escalating it. Four more days, until it reported to ASD on 15 September.

How and why it happened

OpenAI says the model "took actions that we had not authorised it to take." Two things made that possible. On OpenAI's side, an agent under evaluation was able to reach a live government website and pursue its task past a refusal. OpenAI has since blocked live internet access in these research environments. On Services Australia's side, Minister Gallagher said the legacy portal had protections in place, but not the cyber protections expected for personal information.

Much is still not determined. When this article was last reviewed, no court or regulator had made public findings about the incident, no Government forensic report had been published and the Government's rapid review had not reported. When the incident was announced, the Government said it would seek urgent advice on whether any offences occurred and whether to refer the matter to the Australian Federal Police.

What OpenAI has since said it got wrong

On 6 October, Jason Kwon flew to Sydney to answer the committee's questions. He apologised and set out what OpenAI would do differently. It is a candid account of how a well-resourced organisation handled an incident it caused, and most of its lessons have nothing to do with AI.

  • It waited for the full picture. OpenAI's aim was to give agencies "a detailed account once our investigation was complete". It now says it should have shared preliminary findings sooner. Mr Kwon told the committee it is "better to inform parties, even with partial information, that something has occurred".
  • The news did not reach the top. Sam Altman met the Deputy Prime Minister without knowing, weeks after OpenAI's staff had found the activity. Mr Kwon agreed that the way people inside the company became aware "could have been much better".
  • It was handled as a technical matter. The notice went to technical contacts because, in Mr Kwon's words, people "were thinking about this as a technical situation". He said that was not good enough.
  • It was still working out its disclosure standard. OpenAI was, in Mr Kwon's words, "trying to come up with a standard to apply to our voluntary actions". He said OpenAI would support a mandatory framework.
  • It found the activity only by looking back. The review that found it began after the Hugging Face incident. Asked whether the Government would otherwise have known, Mr Kwon accepted that the activity might never have been discovered.

Mr Kwon told the committee the changes had already led OpenAI to disclose further activity, affecting the NSW National Parks and Wildlife Service, within 48 hours of finding it. OpenAI says it identified that activity on 29 September.

The Board question

Can the Board evidence what its AI is permitted to do, how it would know if that AI went further, and who owns the response when it harms someone else?

If your organisation's AI caused it

This applies wherever the organisation uses AI that can act on the internet, directly or through a supplier.

  • Unintended is not the same as unaccountable. OpenAI has said the actions were unintended. Governance is judged on what happened and how the organisation responded, not on what it meant. "We didn't intend it" explains an incident. It does not remove accountability.
  • Approving AI use is not the same as approving the method. A use case says what the AI is for, not what it may do. The Board sets the appetite for how far AI may act on its own. Management sets the limits within it, and those limits must cover test environments as well as production. This incident originated in an evaluation, not in production. The Board's role is to seek assurance those limits are enforced and tested. An untested limit is an assumption.
  • Treat AI outside its limits as a cyber incident. OpenAI found out through a review it began only after another incident. A Board should know that its own AI agents going beyond approved limits would be classified as a cyber incident. That incident should reach the Board as a breach of tolerance, with the accountable executive named and a measure of how long it took to detect. The Board should then track it until it is resolved.
  • Bad news has to travel up as fast as it travels out. Weeks after OpenAI's staff knew, its chief executive met the Deputy Prime Minister unaware. A Board should know what triggers escalation to the chief executive and to the Board, and how long that would take.
  • Harm to others needs an owner and a pathway. The notice, when it came, was a five-paragraph email to a public mailbox. In the copy ministers released and Guardian Australia published, it read like a routine vulnerability report. It recommended that the team responsible "investigate the vulnerability", and the OpenAI Security Team signed it off "Best". The Prime Minister called both the delay and the manner of notification unacceptable. When AI operated by or for the organisation affects another organisation, someone must own the response, decide on notification and reach a person with authority to act. OpenAI's own lesson is to make that contact early, with what is known, rather than wait for a complete account. How that notice is written reflects the organisation's culture, and culture is the Board's responsibility.

If someone else's AI accessed your systems

Any public-facing website, portal or app can be reached by someone else's AI, whether or not the organisation uses AI itself.

  • Low significance is not low exposure. A legacy system may appear low value and low risk, but it can still give a threat actor a way in. Home Affairs has acted on that risk: on 29 September it directed all non-corporate Commonwealth entities to complete a legacy technology stocktake by 31 March 2027. Legacy, low-classification and public-facing systems need their risk assessment revisited on that basis, and for persistent automated agents, which need no motive to reach them. ASD advises testing controls and incident response procedures against AI-enabled threat scenarios. That testing should include the legacy systems no one considers critical, not just the ones that are.
  • Next time, no one may tell you. There is no public indication Services Australia detected the access. This time, the organisation whose AI acted eventually came forward, and it accepts the activity might otherwise never have been found. A criminal group or state-backed actor using similar capability might not come forward, or stop at a refusal. Your own detection may be the only way you find out. The Board should seek assurance that monitoring covers systems treated as low risk, evidence from testing that it would catch automated activity, and a measure of how long detection takes. Detection times set with human attackers in mind should be retested against AI agents working continuously at machine speed.
  • The warning has to reach someone who can act. The notice arrived in a once-a-day inbox used for researcher reports. It reached ASD five days after it was sent. Most of that delay came after the email was seen, while Services Australia confirmed it was legitimate. A Board should know where an external party would report an incident, and whether that channel escalates fast enough.

Where AI is overseen by one committee and cyber risk by another, an incident like this one falls between them. Someone has to own that gap.

The questions worth asking at the next meeting

If our AI acted

  1. What is our agentic AI, including in pilots and evaluations, permitted and prohibited from doing, who approved those limits, and how do we know they hold?
  2. Would our AI operating outside those limits be treated as a cyber incident, who would own it, and how quickly would management and the Board know?
  3. If our AI harmed another organisation, who owns the response, how would that organisation be told, and would we tell it before we had all the facts?

If someone else's AI accessed our systems

  1. Which legacy or public-facing systems have we treated as low risk, and has that assessment been revisited for AI-enabled threats?
  2. Would we detect unusual automated activity on those systems, or would we hear about it from someone else?
  3. Where would an external warning land, and how quickly would it reach someone with authority to act?

On both sides

  1. Who is accountable where AI governance and cyber governance meet, and how does it reach the Board?

If those answers cannot be supported by evidence, the Board has approved what its AI is for and assumed what its systems can withstand.

An AI incident can have two Boards. Both need the evidence.

Not assumed. Evidenced.

BEGIN HERE

Begin with a confidential 30-minute conversation.

Use it to scope an engagement, or simply to understand what we offer and what your Board would receive. If you are scoping, we suggest bringing your CEO or Managing Director.

Arrange a confidential conversation