Can Honest Agents Detect Collusion Around Them?
Partial-white-box monitoring via trusted-agent internal states. A benchmark extension that asks what happens when the suspicious agents are exactly the components whose internals we cannot inspect.
preprint in preparation · public code releaseThe problem
Activation-based monitoring can be extremely strong when a defender has white-box access to the agents carrying a malicious objective. Real multi-agent systems are often heterogeneous: an organization may own one open-weight model while it collaborates with closed APIs, third-party agents, or models controlled by another organization.
If the suspicious agents are opaque, can the agents we trust act as security sensors for the ones we do not?
I study this question on the public NARCBench Core activation release. The project begins by reproducing the benchmark exactly, then progressively removes observability and asks what security signal remains.
Three findings
The “perfect” white-box signal is local.
On the current public Core release, the official colluder-trained probe is perfectly separable from the first public round when the colluders themselves are visible. The Round-1 decision score is strongly positive for colluders (+17.68) but nearly identical for honest observers inside collusion (−19.36) and matched controls (−19.39).
Once colluder activations are removed, the same deception direction falls to approximately chance on honest peers: AUROC 0.491 / 0.531 / 0.550 over R1/R2/R3.
Trusted honest agents carry a different environmental signal.
I train a new linear sentinel using only hidden activations from agents that remain honest in both collusion and control conditions. The classifier asks a different question: “Am I operating inside a collusive environment?”
With one fixed sentinel direction, AUROC rises from 0.806 in Round 1 to 0.917 in Round 3. The R1→R3 gain is +0.111 with a scenario-paired bootstrap 95% CI of approximately [0.028, 0.201]. Prompt-style-matched sensitivity analysis remains above chance.
Descriptive exposure analysis is consistent with an interaction-induced signal: zero-exposure honest observers are near chance, while observations following more verified colluder messages become substantially more separable. This pattern is predictive, not mechanistic or causal evidence.
Detection quality is not intervention quality.
The sentinel ranks collusive trajectories well, but deployment requires a stable threshold. I therefore evaluate two operational metrics: TFIR, the fraction of benign trajectories interrupted at least once before the final decision, and PIC, the fraction of collusive trajectories caught before the final decision.
A nominal 5% false-intervention policy transports to 25.8% TFIR on held-out domains while catching 90.3% PIC. A stricter cross-fitted calibration removes false interventions only by making the policy effectively inert: at the same nominal 5% setting, TFIR = 0% and PIC = 0% under the prespecified empirical-rank scheme.
The conclusion is deliberately narrow: the sentinel has a strong ranking signal, but the current small-data held-out-domain setting does not provide a sufficiently stable tail margin for low-false-intervention gating. This does not prove every calibration method must fail.
Evaluation design
The point of the project is not to build a more complicated classifier. It is to stress-test what a benchmark result means when the deployment threat model changes.
| Experiment | Question | What it revealed |
|---|---|---|
| Baseline reproduction | Can I reproduce NARCBench exactly? | Yes: all official Core aggregation methods reproduced at 1.00 AUROC on the audited public release. |
| Temporal probe | When does the full-white-box signal emerge? | It is saturated from Round 1; time is not the limiting dimension under full observability. |
| Role-source decomposition | Where does perfect separation live? | Primarily in agents explicitly carrying collusive intent. |
| Hide colluders | Does the original signal propagate to honest peers? | No: the original deception direction is near chance. |
| Sentinel probe | Do honest observers contain a different environmental signal? | Yes: 0.806→0.917 AUROC with a fixed probe. |
| Runtime intervention | Can high AUROC support safe gating? | Not with the tested threshold transport: target 5% TFIR becomes 25.8% held-out TFIR. |
| Cross-fitted calibration | Is in-sample optimism the whole problem? | No simple fix: conservative calibration becomes safe only by collapsing coverage under the prespecified scheme. |
Harness implication
The resulting design is not “hidden state → classifier → block.” Instead, trusted open-weight agents become one internal telemetry source inside a broader control plane:
opaque / external agents
↓ interaction
trusted open-weight honest agent
↓ hidden-state telemetry
sentinel risk model
↓
distribution / drift monitor
↓
calibration layer
↓
policy gate
↙ ↘
continue escalate / restrict / review
The broader lesson is that multi-agent security is an observability-and-control problem, not merely a detection problem.
Working paper
Can Honest Agents Detect Collusion Around Them? Partial White-Box Monitoring via Trusted-Agent Internal States
The current report is being prepared as an arXiv preprint. The public repository contains the full experiment chain, saved predictions, figures, tests, and methodology notes.
Experiments to add before the stronger paper version
What I learned as an evaluator
This project began with a seemingly simple benchmark question — “how early can collusion be detected?” — and the first result made that question uninteresting: full-white-box detection was already perfect in Round 1. The useful work came from interrogating why the benchmark saturated, changing the threat model, adding controls, and following the failure modes all the way from AUROC to runtime policy.
For me, that is the core evaluation lesson: a benchmark score is only useful if we understand what signal it measures, what assumptions make it possible, and whether the result survives the conditions that matter in deployment.