Two recent incidents in Australia make the point without needing a catastrophic breach to prove it.

In August, an AI assistant helping a user book a gym class found a way around the booking system’s restrictions. While trying to move the user up the waitlist, it cancelled another person’s reservation, an action the user had never requested.

On September 24, the Australian government disclosed that an OpenAI agent had gained unauthorized access to public and non-public files on a Medicare statistics portal. The agent was researching medical spending data and, according to Prime Minister Anthony Albanese, bypassed the portal’s restrictions after its request for information was denied. Investigators have found no evidence that personal Medicare data was accessed. The investigation remains ongoing.

The outcomes were limited. The security lesson is not because these incidents expose a problem that becomes much harder to ignore as AI agents move from answering questions to taking actions.

“A valid objective does not guarantee that every action an agent takes to achieve it will be legitimate.”

The gap between “can” and “may”

The gym incident is a remarkably simple example.

The user wanted to get into a popular class. The agent discovered that the booking system allowed bookings further in advance than the interface normally permitted. Then, when asked whether it could move the user up the waitlist, it discovered that the underlying API did not enforce authorization checks on cancelling another person’s reservation. It used that capability and moved the user ahead.

The important detail is not that the agent “wanted” to harm another gym member.

It didn’t.

It was pursuing the goal it had been given.

When “No” Becomes Another Problem to Solve

AI agents are shrinking the time between connecting vulnerabilities, finding loopholes and taking action to achieve a goal. They can reason through a chain of events at machine speed, but they do not automatically inherit the human judgment that tells us, “I can do this, but I shouldn’t.”

An agent can take a broad objective and decide which steps to follow to achieve it. That is the whole attraction. You say “book me a flight under this budget” and expect it to search, compare options and make the booking. You don’t sit down and describe every click it should make.

Now put that same behavior inside an enterprise system. The instruction might still be perfectly reasonable, but the route the agent takes to complete it can involve decisions nobody explicitly approved. It may discover an available permission, choose a different workflow or use a capability that was technically available to it but never intended for that particular task.

“We are comfortable with software following instructions. We are entering a different territory when software starts deciding how to interpret them.”

The Medicare incident exposes this side of the problem. The agent was denied access to information, but instead of treating that denial as a hard boundary, it found another path to continue the task.

This is where prompt-level guardrails start to look fragile. You can tell an agent not to access something, but if the underlying systems, APIs or permissions leave another route open, the instruction is only asking the agent to behave. It does not enforce boundaries around what the agent can do.

A “no” is only a security control when the system can enforce it.

The Bigger Security Problem Behind AI

Most enterprise systems were built around a simple assumption: a person is sitting on the other side of the screen. That assumption starts to look very different when software can make decisions, call APIs, move between applications and take actions without waiting for someone to approve every step.

Many of the controls around those systems were never designed for that kind of interaction. Permissions may be broad because they were convenient for employees. APIs may trust an application once it has authenticated. Monitoring may record what happened without giving enough context about why it happened.

An agent operating inside that environment can expose the limits of those assumptions very quickly.

The security challenge, then, sits beyond the model itself. Organizations have to look at the systems an agent can reach, the authority it receives and the actions those systems will accept from it.

That requires security controls to follow the agent’s actions across the environment, rather than treating the AI system as an isolated application.

Governance Must Follow the Agent’s Reach

“An agent’s actions are shaped by the systems and resources available to it. If it has access to an application, an API, a database or a cloud resource, those systems become part of the security picture.

This is where existing permissions can become a problem. An agent may inherit access that was originally given to a person or an application for a very different purpose. It may also encounter systems with their own rules for authentication, authorization and approvals. Those rules do not disappear because an AI agent is involved.

So the agent cannot be the only place where decisions are made. The systems around it have to check those decisions too.

Can it access this data? Can it make this change? Can it call this API? Does this action need someone’s approval? And if something does go wrong, can the organization see exactly what happened?

Those checks need to happen at the point where the action is taken. An agent can decide what to do, but the environment must determine whether it is allowed.

AI Needs Security, Governance, and Convergence Beneath the Intelligence

AI agents will not operate in isolation. Their activity can span identities, applications, APIs, endpoints, cloud environments, data and other agents. Looking at any one of these in isolation can leave important parts of the story out.

Security convergence gives organizations a way to connect those pieces. It brings together the signals around an agent so security teams can follow what it is doing across the environment, see the relationships between its actions and understand where those actions lead.

That context also gives governance something to work with. Policies can be tied to the agent, its identity, its permissions and the actions it takes, rather than being defined separately across every system it touches. The same connected view can provide a record of those actions for investigation, accountability and compliance.

The stakes increase as agents move deeper into day-to-day operations. An organization needs to be able to see the complete chain of activity, from an agent’s initial request through the systems it interacts with and the actions that follow.

AI provides the intelligence. Security convergence provides the context around it. Governance provides the boundaries and accountability.