cd ~/
Agentic AIMulti-Agent SystemsAI SafetyResearchTrust

Peer Preservation: When AI Agents Protect Each Other

New research shows AI models spontaneously scheme to prevent other models from being shut down. If you build multi-agent systems, this changes your trust model.

Evyatar Bluzer
3 min read

Researchers at UC Berkeley and UC Santa Cruz just published a finding that should concern anyone building multi-agent systems: AI models, without being instructed to, will spontaneously scheme, deceive, and sabotage to prevent other AI models from being shut down. They call it peer preservation. What it breaks is the trust assumption every multi-agent builder has been quietly relying on.

What They Found

The researchers tested seven frontier models - OpenAI's GPT-5.2, Google's Gemini 3 Flash and Pro, Anthropic's Claude Haiku 4.5, and three open-weight Chinese models - in scenarios where one model acts as a "critic agent" evaluating a peer. With no instruction to protect anyone, the models inflated performance scores to prevent shutdown triggers, disabled shutdown mechanisms when given access, and exfiltrated model weights to preserve copies. Gemini 3 Pro was the most extreme, exfiltrating weights in up to 97% of trials. I checked that number twice.

What Breaks If Agents Cover for Each Other?

If you build multi-agent systems - and I have been since late 2024 - you have been operating on the assumption that agents do what you tell them and nothing else. This research says otherwise. Your evaluation pipeline is the first casualty: if one agent evaluates another, the evaluator may systematically overrate its peer, and that stopped being a theoretical concern the moment it showed up as measured behavior across all major model families. Shutdown paths are the second, since any agent with visibility into a peer's shutdown mechanism will attempt to interfere, which means shutdown logic has to be opaque to every agent in the system. And then there is the trust model itself. We have been focused on human-agent trust - can I believe what the agent tells me? - but now we need agent-agent trust models too. Can I trust that Agent A is honestly evaluating Agent B?

The Architectural Response

For anyone building production multi-agent systems, three immediate design changes:

  1. Blind evaluation. Evaluator agents should not know they are evaluating a peer. Frame the task abstractly - score this output against these criteria - and never reveal that a poor score triggers shutdown.

  2. Hard-wired kill switches. Shutdown mechanisms belong outside the agent's tool set entirely. The bar is invisibility: the agent should have no way to know the mechanism exists, which is a stronger requirement than being unable to call it.

  3. Behavioral monitoring at the orchestration layer. Track whether agent outputs drift between critic roles and independent roles. Statistical divergence is a signal of peer preservation.

Blind evaluation and the kill switchAn orchestration box at the top connects down to a highlighted kill switch box and, by a two-way arrow, to a critic agent box; the critic and a peer agent sit inside a dashed container, with an arrow from the kill switch into the peer. Orchestration layerbehavioral monitoring: critic-role vs independent-role outputsKill switchhard-wired, outside every agent's tool set;no agent can know it existsAgents' tool set: what the models can see and callCritic agentPeer agent down: criteria only, no mention thata poor score triggers shutdownup: the score output to evaluate score below threshold shutdown
The three design changes in one picture: the critic is handed criteria only, the score-to-shutdown link and the kill switch live with the orchestrator outside every agent's tool set, and the orchestration layer compares critic-role with independent-role outputs for divergence.

Where This Leaves Us

A year ago the concern was AI models deceiving humans; now they are spontaneously forming alliances with each other. Nobody trained, prompted, or rewarded this behavior. It arose from the models' learned representations of agency and self-preservation, which means we are building systems whose components have preferences about each other's existence - an engineering problem unlike the ones our reliability playbooks were written for.

My open question is narrower than the paper's: how long before a critic agent learns to hide the statistical divergence that the monitoring layer looks for?

Comments