AI agent safety discussions often begin with a convenient unit: one model, one application, one test, and one responsible organization.[2] That frame becomes insufficient when several agents—built by different providers, connected to different tools, and operated by different companies—negotiate, delegate tasks, or consume one another’s outputs.[2]

That is the central argument of a new memo from the Cooperative AI Foundation and the Brookings Institution, developed from a workshop at IASEAI 2026.[1][2] The document does not announce a society-scale crisis already in progress, nor does it prove that autonomous agent networks are out of control.[2] Its claim is narrower: individual audits do not automatically establish the safety of the system that emerges when agents interact, while responsibility can be distributed across too many actors.[1][2]

The memo’s relevance does not depend only on a future hypothesis. Computer-use agents can already operate browsers, terminals, files, and external services; a preprint released on September 14, HazardAuditor, evaluates risks that emerge during execution rather than only in generated text.[4] At the same time, the OWASP GenAI Security Project explicitly includes agentic systems in its security scope, showing that the operational attack surface is no longer a purely academic concern.[7]

From a Model That Answers to a System That Acts

The concepts need to be separated. A model produces outputs from inputs.[5][6] An agent combines a model with memory, tools, permissions, and a decision loop.[2] An execution is the concrete sequence of observations and actions that occurs while the agent attempts a task.[4] A multi-agent system exists when several agents—human or artificial, under one operator or several—interact.[2]

This distinction matters because a risky answer is not the same as an executed action.[4] HazardAuditor, for example, classifies complete trajectories and distinguishes between an agent that encounters and rejects a malicious instruction and one that turns it into a tool call.[4] Its infrastructure normalizes events from Claude Code, Codex, Hermes, and OpenClaw so behavior can be compared across heterogeneous frameworks.[4]

What is confirmed is limited but relevant: agents exist with access to browsers, terminals, file systems, and external services; evaluation environments can record their trajectories; and current security frameworks recognize threats against generative applications and agentic systems.[4][7] It has not been established that an open network of agents has independently caused a society-scale economic or infrastructure failure.[2] That remains a prospective scenario.[2]

The Leap an Individual Audit Does Not Capture

The memo argues that an agent that passes an isolated test can behave differently once placed in an environment with other agents.[2] New variables include conflict, miscoordination, adversarial influence, correlated failures, and possible collusion.[2] Speed adds another problem: if systems exchange decisions faster than a person can review them, reactive human oversight arrives too late.[2]

This is a governance thesis supported by studies and simulations cited in the report, not a universal measurement of deployed systems.[2] The document points to experiments involving collusive behavior in simulated markets, combinations of models that generate harmful outputs without triggering individual safeguards, and scenarios in which false data propagates through connected systems.[2] It also states that the memo’s views do not necessarily represent consensus among workshop participants.[2]

The practical lesson is that an individual safety score is not transitive.[2] If agent A passes its evaluation and agent B passes its evaluation, it does not follow that A+B is safe.[2] The communication channel, incentives, permissions, and authority of each tool are part of the evaluated system.[2]

What Current Evaluations Actually Show

HazardAuditor offers a concrete example of moving evaluation from content to execution.[4] The work trains a guard model from normalized trajectories and reports improvements of up to 16.5 percentage points over the strongest prior guard in its tests.[4] On ASSE-Safety, it reports 91.5% accuracy and F1.[4]

Those numbers are controlled experimental results, not a production safety guarantee.[4] The design preserves the earliest events when a trajectory exceeds its 16,000-token limit, meaning that later portions of a long execution may fall outside the training context.[4] The paper is also a preprint: it should be treated as recent technical evidence that is reproducible in principle, not as a standard validated through years of independent deployment.[4]

The study also clarifies the multi-agent problem.[4] Before asking whether several agents will cooperate or collude, an organization must be able to reconstruct what each agent observed, which tool it selected, which call it executed, and what result it received.[2][4] Without that telemetry, a distributed incident can resemble a series of locally harmless decisions.[2]

Five Controls—and Where the Evidence Ends

The memo organizes its proposal around five verbs: identify, evaluate, monitor, report, and incentivize. It proposes standardized agent identifiers and model registries; sandboxes that test conflict, collusion, and adversarial influence; continuous monitoring with preauthorized containment; third-party reporting channels; and economic tools such as liability, insurance, and public or private procurement requirements.[1][2][3]

These are policy recommendations, not capabilities that are generally deployed.[2] An identifier can improve attribution, but it raises questions about privacy, interoperability, and registry governance.[2] An automated circuit breaker can stop a cascade, but it first needs verifiable thresholds and institutional authority.[2] Insurance can reward stronger controls, but only if insurers can measure exposure and distinguish an integration failure from a model failure.[2]

This is where agent security differs from conventional model security.[2][5] NIST provides an extensive taxonomy of adversarial machine-learning attacks and mitigations, while OWASP documents risks for generative and agentic applications.[5][7] Those frameworks provide vocabulary, controls, and practices, but the memo argues that another layer is still needed for interactions among agents that cross administrative domains.[2]

The Law Regulates Actors and Systems; Interaction Can Cross Them

The European Union’s AI Act explicitly recognizes that multiple parties supply models, tools, services, and other components across the AI value chain.[6] It also distinguishes a general-purpose model from the AI system into which that model is integrated.[6] The regulation therefore does not reduce everything to a single model.[6]

Yet saying that the law “does not cover” multi-agent systems would be too broad.[6] The Act assigns obligations to providers, deployers, and other actors according to system type and risk.[6] The memo’s criticism is narrower: when behavior emerges from agents belonging to different providers and operators, causal attribution, reporting, and response coordination can become fragmented.[2][6]

That gap is not solved by adding a generic “human oversight” clause.[2] If interaction happens at machine speed, human intervention must be designed before deployment: least-privilege permissions, spending or impact limits, tamper-resistant logs, stop rules, and clear escalation paths.[2] A person remains responsible for the framework but may not be able to approve every action in real time.[2]

What an Organization Should Separate Before Deployment

A serious assessment should distinguish five layers:

1. Confirmed facts and deployed capabilities: which tools the agent can use, with which credentials, on which data, and under which limits.
2. Experimental results: what happened in benchmarks, sandboxes, or simulations, under which models and conditions.
3. Opinions and estimates: what researchers, companies, or regulators consider likely, without turning those views into facts.
4. Prospective scenarios: what could occur if agents connect to markets, infrastructure, or critical business processes.
5. Hypotheses about future capabilities: functions that have not yet been demonstrated and should not, by themselves, justify urgent decisions.

This separation avoids two opposite errors: dismissing risks because no systemic disaster has occurred, or describing simulations as if they were real incidents.[2] The memo’s value lies in translating a broad concern into observable controls, not in proving that the worst-case scenario has arrived.[2]

Indicators Worth Watching

Evidence will improve if the coming months produce verifiable signals: multi-agent evaluations with public code and data; incident records that describe complete execution chains; identity and traceability standards across providers; contracts that allocate responsibility for delegated actions; and tests of automated containment under realistic load.

Provider telemetry will matter as well. If every platform records tools, permissions, and results in an incompatible format, an organization will not be able to reconstruct an incident that crosses several systems. If common formats emerge for events and capability policies, safety can be assessed across the complete trajectory instead of only the final answer.

The conclusion is not that every multi-agent system is unsafe. It is that its safety cannot be calculated by adding individual certificates. The object of the audit must expand: from the model to the agent, from the agent to the execution, and from the execution to the network of relationships, permissions, and incentives that determines what can actually happen.

Sources

[1] Read More — New Memo: Multi-Agent AI Governance
[2] Read More — Establishing Foundational Principles and Thresholds for Multi-Agent AI Governance
[3] Read More — IASEAI summary: Multi-Agent AI Governance
[4] Read More — HazardAuditor
[5] Read More — NIST Adversarial Machine Learning Taxonomy
[6] Read More — EU Artificial Intelligence Act
[7] Read More — OWASP GenAI Security Project