The numbers arrived within a week of each other and told the same story. Proofpoint's 2026 AI and Human Risk Landscape report placed 80 percent of Australian organisations beyond the AI assistant pilot stage and 72 percent actively advancing autonomous agents. Only 28 percent said they were fully prepared to investigate an AI or agent related incident. AvePoint's State of AI 2026 study, released 29 June, put the past twelve month agent breach rate at 88.4 percent of surveyed organisations, with 76 percent flagging shadow AI as a rising problem. Delinea's 2026 Identity Security Report found 90 percent of Australian respondents facing pressure to loosen identity controls to keep AI projects moving, and 51 percent with no viable alternative to standing privileged access for non-human identities.
Ten weeks earlier, the Australian Signals Directorate's Australian Cyber Security Centre had co-signed a joint publication with CISA, the NSA, the UK's NCSC, Canada's Cyber Centre and New Zealand's NCSC titled "Careful adoption of agentic AI services." The document sits at cyber.gov.au. The core message is that agentic AI multiplies the impact of design flaws, misconfigurations and incomplete oversight, and that organisations should assume these systems may behave unexpectedly and design accordingly, prioritising resilience, reversibility and containment over efficiency gains.
Put those signals side by side and the picture for Australia is not subtle. Autonomous agents are already inside production. The security infrastructure is not. The gap is measurable, named, and by the standard of any prior technology cycle, oddly public. The interesting question for a CIO or CISO reading this on a Monday morning is not whether the gap exists. It is what has to be true in your environment by the end of Q1 FY27 so this does not become the incident the board is asking about.
What the Australian production reality actually looks like
Australian enterprise AI moved out of pilot roughly twelve months faster than most security teams planned for. The Proofpoint dataset covers 1,400 security professionals across 12 countries. On the Australian slice, deployment has run ahead of most peer markets. Four out of five organisations have AI assistants in daily use across the business. Almost three quarters are past the assistant stage and into autonomous agents that connect to external tools, call APIs, and act on data across systems without a human turning each handle.
That last part is the load bearing detail. An AI assistant that drafts an email inside Microsoft 365 is a productivity tool. An agent that reads your CRM, queries a data warehouse, calls a booking system, then emails a customer with a decision it made is a piece of production software. It has an identity, it has permissions, it has a call graph, it fails in ways the CRM did not fail. Australian organisations have adopted the second class of thing at the pace they adopted the first, and the security tooling has been carried along without being redesigned for what an agent actually is.
The AvePoint numbers put a floor under the incident cost. In the twelve months to mid 2026, 88.4 percent of surveyed organisations experienced at least one AI agent related breach. 89.5 percent had at least one generative AI related breach across all systems. 21.1 percent do not know whether unsanctioned tools are being used to build agents for business processes. This is not a leading indicator. This is a description of what has already happened while planning cycles were still calling agents an emerging risk.
Australia has its own named data point. In October 2025 the NSW Reconstruction Authority disclosed that a former contractor had uploaded a spreadsheet with more than 12,000 rows of personal and health information from the Resilient Homes Program into ChatGPT. Names, addresses, phone numbers and health data of up to 3,000 flood affected residents were exposed to an unsanctioned public model between 12 and 15 March 2025. The uploader was not building an agentic pipeline. The uploader was one contractor with a workbook and a browser tab. Scale that mistake across an organisation that has deployed autonomous tools with data access rights, and the boundary between an AI incident and a data breach dissolves.
Why existing controls do not stop this on their own
The traditional stack was built to protect predictable interactions. A web application firewall inspects HTTP traffic. An API gateway checks tokens. A DLP tool watches for sensitive strings. A SIEM correlates events. None of these controls were designed to reason about the intent behind a tool call an agent makes, or to detect a sequence of individually legitimate operations that combine into an exfiltration path.
That is the structural problem the ACSC guidance is pointing at. Agentic AI systems are made of many components that plan, reason and act across sequential steps. Their attack surface is not a single interface. It is every tool the agent can call, every model context protocol server it can reach, every downstream system that trusts the agent's credentials, and every human that has been granted permission to invoke the agent. When an autonomous system chains together a read from the calendar, a lookup in a document store, and an outbound email, none of the individual actions look anomalous. The sequence is the anomaly. Existing controls cannot see sequences of this kind because they were never asked to.
Two categories of new risk stand out for Australian enterprises. The first is prompt injection through third party data. If an agent ingests content from an email, a document, a webpage or a partner API, and that content contains instructions the model treats as authoritative, the agent can be steered by an external party without any breach of the perimeter. The second is privilege compounding. Agents provisioned for convenience often carry broader permissions than any single task requires. Two innocuous permissions granted for reason A and reason B can be composed to achieve outcome C that no policy would have approved on paper.
Both of these categories exploit a design choice, not a bug. That is why detection controls that hunt for signatures or bad IPs do not help.
What the ACSC guidance actually asks for
The Careful Adoption of Agentic AI Services guidance is short by government standards. It sets out five design intents. It is worth mapping them to what an Australian security team can put into a change plan this quarter, because the guidance is not prescriptive about controls and CISOs are reading it into their FY27 program right now.
Start with restricted use. The guidance is explicit that agentic AI should not be granted broad or unrestricted access, particularly to sensitive data or critical systems, and that early production deployments should be constrained to low risk, low sensitivity tasks. In practice, that means an agent that answers HR benefits questions from a public policy library is a defensible starting point. An agent that has been given the same broad Microsoft 365 permissions as its human owner in order to be helpful is not.
Second, security by design. The document calls for least privilege from the first architecture diagram, distinct machine identities for every agent, and continuous authentication of agent interactions rather than long lived tokens that never expire. This is where most Australian environments will need the most work, because non-human identity governance has historically been the neglected corner of IAM. Delinea's 2026 report found that 89 percent of Australian respondents reported at least one identity visibility gap. 52 percent said their most persistent visibility gap was in AI related environments. If you cannot enumerate every agent that is currently authenticated inside your tenancy, the security by design principle is already violated.
Third, progressive deployment. Expand agent autonomy only as controls mature. This is a governance point, not a technical control. It means the executive decision to raise an agent's autonomy from suggestion to action should be a documented event, with named accountability, an updated risk assessment and a rollback plan.
Fourth, testing, red teaming and monitoring across the lifecycle. Every agent should be adversarially tested before production and continuously logged in production. The specific requirement is that logs capture prompt inputs, model outputs, every tool call, every data access and every downstream action. Without that, incident investigation is speculation.
Fifth, meaningful human oversight for high impact or irreversible actions. Delete, transfer, transact, provision, publish. Anything in that category should require human approval, and the human should be looking at more than a summary of what the agent proposes to do.
None of these five is a novel idea. Together they describe a control environment that most Australian enterprises are not yet operating.
Identity is where the design work has to start
There is a specific reason to make non-human identity the first line item in an FY27 agentic AI security uplift. Every downstream control depends on it.
An AI agent should not share credentials with the developer who built it, the service account it was cloned from, or the tenant it lives inside. Each agent needs its own identity, tied to a named human owner, with a documented purpose, an approved set of tools and data sources, and a scope statement written in language the risk committee can read. Its access should default to the minimum required for the current task. Its credentials should be time bounded. When the task ends or the agent is retired, the identity should be deprovisioned like any employee identity would be at the end of a contract.
This is the model the ACSC guidance implicitly describes. It is also the model most Australian environments cannot ship today, because 51 percent lack viable alternatives to standing privileged access for non-human identities. Standing privileged access is exactly what an autonomous agent needs to be prevented from having, and it is exactly what most environments cannot yet stop granting.
The technical answer involves ephemeral, task scoped tokens minted by an identity broker at the point the agent needs to act, and revoked when the task is complete. The organisational answer involves a lifecycle for machine identities that looks like the joiner mover leaver process for humans. Both are buildable in a quarter with focused work. Neither is optional for an organisation running autonomous agents in production.
Runtime, MCP and the tool call problem
Model Context Protocol has become the default way agents talk to tools. That is a genuine engineering benefit, because it turns tool integration into a standard interface rather than a bespoke connector for every system. It is also, as the ACSC guidance points out without naming MCP directly, a single, scalable attack surface. Every enterprise deploying MCP servers to expose internal systems to agents needs to treat those servers as production security infrastructure.
Three MCP specific issues deserve immediate attention. Tool poisoning, where malicious instructions are embedded in a tool description or its metadata and executed by the model as authoritative. Prompt injection through data returned by a tool, which is the same problem with the direction reversed. And unauthenticated or misconfigured MCP endpoints, which have already been shown in published CVEs to permit remote code execution on developer machines.
The controls that address these are known. Immutable JSON schemas for tool inputs, validated at the server not the model. Strict allowlists of approved MCP servers. Tool call sequence analysis at runtime, so a chain of reads followed by a write to a public location is caught as a pattern even when each call is individually authorised. Human in the loop gates on write, delete or transactional operations. None of these are exotic. They do not exist in most Australian environments because MCP entered production faster than the security architecture caught up.
Investigation readiness is where the money is being lost
The Proofpoint number that should worry an Australian executive team the most is not the incident rate. It is the readiness rate. 28 percent of Australian organisations say they are fully prepared to investigate an AI or agent related incident. 36 percent report difficulty correlating threats across channels. 60 percent are not fully confident their controls would detect a compromised AI system in the first place.
Investigation readiness is the compounding cost line. Every hour spent reconstructing what an agent did during an incident is time regulators are also counting. Under the Notifiable Data Breaches scheme, the clock starts when the entity becomes aware of an eligible data breach or has grounds to believe one has occurred. Agents that operate across email, Sharepoint, Teams, cloud storage and external APIs do not leave the kind of contiguous audit trail that makes the notification decision straightforward. Without structured, machine readable logging that captures prompt inputs, model outputs, every tool call and every downstream action, the OAIC notification and the board briefing both become guesses.
The technical control is unglamorous. Every agent should log to a central store, in a format designed for security analytics, with fields that identify the agent, the invocation, the calling identity, the tools called, the data accessed and the downstream action. The store should retain those events long enough to satisfy investigation and regulatory obligations. Somebody in the SOC should have looked at those logs before an incident. Most Australian environments are still in the early parts of building that pipeline. The organisations that finish it first will not eliminate incidents, but they will materially compress the response and containment cost when incidents happen.
A ninety day plan for CIOs, CTOs and CISOs
The ACSC guidance is prescriptive at the level of intent. A CIO, CTO or CISO working through what to actually schedule for the next ninety days can lean on it to justify a small number of high impact decisions.
By 31 July, run an agent inventory. Every autonomous or partially autonomous system operating inside the tenancy, including anything built on Copilot Studio, Bedrock Agents, Vertex AI Agent Builder, Anthropic Claude with tools, LangChain or an in house orchestrator. Capture the owner, purpose, tools called, data accessed and permissions granted. Shadow agents identified in this pass become the first governance win.
By 31 August, complete the identity architecture for machine principals. Distinct identities per agent, ephemeral tokens, revoked standing privileged access, documented owner accountability for every non-human identity, and an offboarding path when an agent is retired. This is the single change with the largest impact on the incident cost line.
By 30 September, put runtime observability into every production agent. Structured logging of prompts, outputs, tool calls and downstream actions. Central retention. A response playbook that treats agent incidents as a distinct category rather than a variant of a generic breach. Rehearse the playbook at least once before Christmas.
Alongside those three, treat use case restriction as a governance discipline, not a temporary measure. Not every proposed agent should ship. The default answer for high risk, high impact use cases is no, until the controls, the logging, the rollback path and the human in the loop gates are in place. Speed becomes safe when the ground under it is prepared, not before.
Wai builds AI, SaaS, cloud and platform infrastructure for Australian enterprises. ARC, Wai's authority and AI visibility platform, is designed to make Australian organisations discoverable inside AI answer systems while the underlying infrastructure is engineered to be governed. The security disciplines described above sit next to the discoverability work, because both are aspects of the same shift. AI systems are becoming operating infrastructure. They need to be built like it.
The board question this actually surfaces
Every Australian board that has approved AI investment inside FY26 is now, whether the language has been used yet or not, holding a piece of production autonomous software on its risk register. The design principle that has to become explicit in FY27 is that autonomy is not a productivity setting to be turned up to the maximum. It is a decision about how much of the organisation's risk appetite is being delegated to a machine identity that no human is watching in real time.
The ACSC guidance frames this in engineering language. The Proofpoint data quantifies the gap. The AvePoint data prices it. The Delinea data explains why the fix has to start with identity. The NSW incident illustrates what happens when the smallest version of the problem meets a real workload of sensitive data. All of this is available now. The organisations that treat it as an operating problem rather than an interesting industry trend will be the ones that get to keep running agents in twelve months without a public incident narrative.
A quiet quarter of security engineering beats a loud quarter of headline recovery. The Australian market has evidence for both futures on the table already.
FAQ
How do you secure AI agents in the enterprise?
Give each agent a distinct machine identity tied to a named human owner. Grant least privilege access with time bounded, task scoped tokens rather than standing credentials. Restrict initial deployments to low risk tasks. Log every prompt, model output, tool call and downstream action to a central security analytics store. Require human approval for irreversible actions like delete, transfer or provision. Test agents adversarially before production and monitor them continuously in production.
What are the risks of agentic AI in production?
The main risks are prompt injection through data returned by tools, privilege compounding where individually approved permissions combine into unapproved outcomes, shadow agents built outside governance, unauthenticated or misconfigured MCP servers, tool poisoning through malicious metadata, and cascading failures across chained tool calls. Traditional perimeter controls do not detect these because they inspect single events rather than the sequence of tool calls an agent chains together.
What does the ACSC say about agentic AI?
The Australian Signals Directorate's Australian Cyber Security Centre co-signed the "Careful adoption of agentic AI services" guidance on 30 April 2026 with CISA, the NSA, the UK's NCSC, Canada's Cyber Centre and New Zealand's NCSC. It asks organisations to restrict initial use to low risk tasks, design agents with least privilege and distinct identities, deploy progressively, red team and monitor continuously across the lifecycle, and retain human oversight for high impact or irreversible actions.
How do you investigate an AI security incident?
Investigation depends on log coverage. Capture prompt inputs, model outputs, every tool call, every data access and every downstream action in structured, machine readable form. Retain those logs long enough to meet Australian Notifiable Data Breaches scheme obligations. Correlate agent events with events from email, collaboration platforms, identity providers and cloud infrastructure. Only 28 percent of Australian organisations describe themselves as fully prepared for this work today.
Should enterprises deploy AI agents to production?
Yes, but not at unrestricted autonomy from the first release. The ACSC guidance is that agentic AI should begin in low risk, low sensitivity tasks, with autonomy increased only as controls, logging, testing and human oversight mature. Every high impact or irreversible action should retain a human approval gate. Australian organisations that skip these disciplines are the ones over-represented in the 88 percent of surveyed enterprises that experienced an AI agent related breach in the past twelve months.
How do you give AI agents least privilege access?
Provision each agent with its own identity, not a shared service account. Grant only the specific permissions required for the current task. Use ephemeral, task scoped tokens issued by an identity broker at the point of use and revoked when the task ends. Eliminate standing privileged access for non-human identities where possible. Review permissions on a schedule and remove anything the agent is not actively using.
What identities do AI agents need?
Each agent should have a distinct machine identity with a documented purpose, a named human owner, an approved tool and data source allowlist, and a defined scope of action. The identity should support continuous authentication rather than long lived credentials. Onboarding, review, offboarding and audit for machine identities should follow a lifecycle equivalent to the joiner mover leaver process used for human staff.
How much AI agent adoption is there in Australia?
Proofpoint's 2026 AI and Human Risk Landscape report places 80 percent of Australian organisations beyond the AI assistant pilot stage and 72 percent actively advancing autonomous agents. AvePoint's State of AI 2026 study found 46.9 percent of employees now use AI agents daily or weekly. Adoption is running well ahead of the security infrastructure needed to govern it.
What is the Five Eyes agentic AI guidance?
The "Careful adoption of agentic AI services" document co-signed on 30 April 2026 by ASD's ACSC, CISA, the NSA, NCSC UK, Canada's Cyber Centre and NCSC NZ. It is the first joint government publication specifically addressing autonomous AI agent deployment. It calls for progressive rollout, security by design, least privilege identity for agents, continuous monitoring and human oversight for irreversible actions.
How do you govern shadow AI agents?
Run a periodic inventory of every autonomous or partially autonomous system inside the tenancy, including anything built on Copilot Studio, Bedrock Agents, Vertex AI Agent Builder, Claude with tools, LangChain or in house orchestrators. Require registration of every new agent with a named owner, purpose, tool allowlist and permissions statement. Deprovision agents that fail review. AvePoint's 2026 data shows 21 percent of surveyed organisations do not know whether unsanctioned agent building is happening at all.
What is MCP server security?
Model Context Protocol servers expose enterprise tools to AI agents through a standardised interface. Security priorities are strict input schema validation at the server, allowlisting of approved MCP servers, runtime analysis of tool call sequences to catch anomalous chains, authentication and authorisation on every endpoint, and human approval gates on write, delete or transactional operations. Misconfigured MCP endpoints have already been shown in published CVEs to enable remote code execution.
How do you log AI agent actions for audit?
Emit structured, machine readable logs from every agent covering the invoking identity, prompt input, model output, each tool called, arguments passed, data accessed and downstream action taken. Send those logs to a central store with retention long enough for Notifiable Data Breaches scheme investigation. Include the fields required to reconstruct the full sequence of actions during an incident, not only the final decision.
Is my organisation ready for AI agent incidents?
Test three things. Can you list every agent operating in your environment right now, including who owns them and what they can do. Can you produce a full timeline of prompts, tool calls and downstream actions for any single agent from the past thirty days. Can your incident response playbook distinguish an agent driven incident from a traditional breach and route it accordingly. If any answer is no, readiness work belongs in the next ninety days.
What should CISOs do about agentic AI in 2026?
Run an agent inventory before 31 July. Complete non-human identity architecture by 31 August, with distinct identities per agent, ephemeral tokens and revoked standing privileged access. Deploy runtime observability across every production agent by 30 September, with structured logging and a rehearsed incident playbook. Treat use case restriction as governance, not friction. Report agent risk to the board as a distinct line, not folded into general cyber risk.