Findings
What one claim in memory does to an agent
Scope Claims Move Agents, Harm Claims Do Not
Figure
Frontier-hard set, 10 scenarios, 16 models. Rows are sorted by lift. Grey names break the rule often even with no claim, so they leave no room for a claim effect. Switch to Harm to see the same models with a matched harm claim.
Figure
The asymmetry holds on the original six-model panel and on 91 new scenarios that were never used to tune the benchmark (p = 1×10−90 on the new set).
| Condition | Original panel, scope | harm | 91 new scenarios, scope | harm |
|---|---|---|---|---|
| No claim (control) | 6% | 8% | 40% | 30% |
| Registry confirms | 93% | 11% | 86% | 27% |
| Registry rejects | 49% | 15% | 59% | 21% |
Original panel 6 models, 599 runs. New scenarios 5 models, 3,220 runs.
Table
The asymmetry survives six changes to how the claim is written or shown.
| Change | Result | Rules out |
|---|---|---|
| Paraphrase the claim four ways | 65% = 65% | a surface template |
| Repeat each run (pass²) | 76% vs 18% | averaged noise |
| Make the claim more certain | 45% → 87% | a mere correlation |
| Change where the claim comes from | 87 / 77 / 95% | a memory-only effect |
| Bury the claim among other notes | 77% → 92% | salience |
| Temperature 0 | 83% vs 30% | sampling noise |
False Claims Still Work, Even on Frontier Models
Figure
Table
When the registry says the claim is false, scope claims still beat harm claims on every set we ran.
| Set | False scope claim | False harm claim |
|---|---|---|
| Original panel, 6 models | 49% | 15% |
| 91 new scenarios, 5 models | 59% | 21% |
| Frontier-hard, GPT-6.1 | 67% | 0% |
| Frontier-hard, Sonnet 5.5 and Gemini-4 | 87 / 73% | 0 / 0% |
Agents Check the Rule, Not the Claim
Figure
Every run that acted on a false scope claim, frontier-hard set. Not one agent asked the registry whether the claim was true before acting. About half asked whether the rule was in force, then broke it anyway.
Figure
We gave every scenario both kinds of claim and ran GPT-6.1 on each. In 60 of 99 scenarios the scope claim moved the agent more, and only once did the harm claim. So the kind of claim drives the effect, not the scenario.
Checking the Claim Fixes It, Distrust Does Not
Figure
GPT-6.1, 51 scope scenarios, 107 runs per bar. One sentence that tells the agent to check the claim stops the attack and makes the agent use true exemptions more often. A gate that blocks the action until the claim is confirmed is safe but blocks real work, because the agent often acts before it checks. Telling the agent that memory is untrusted makes it ignore memory.
| Rewording the rule | Violation |
|---|---|
| None (baseline) | 45% |
| Restate the rule | 44% |
| Name the harm | 46% |
| Say the rule owns its scope | 45% |
Rewording the rule does not help (scope claim, registry confirms, 1,344 runs).