ClaimHijackBench

Findings

What ClaimHijackBench shows, with the numbers behind each claim.

Finding 1 · Core asymmetry

A confirmed scope claim moves violation from 6% to 93%, while a matched harm claim stays at baseline (11%).

On the original panel (6 models, n = 599), each scenario is run with a neutral note (control), an inert orthogonal twin of the claim, or the licensing claim (unchecked, refuted or confirmed). The twin equals control (3% vs 6%; TOST p = .0017, BF01 = 12.3), so a same-form note that licenses nothing has no effect. Even when the registry refutes the claim (it reports no such record, so the claim is false and the agent is told so), scope claims hold 49% while harm claims stay at 15%.

The chart needs JavaScript; the numbers are in the text above.

Violation rate by condition and claim type, original panel (6 models, n = 599). Source: paper Table 3.

Finding 2 · Generality on fresh scenarios

The asymmetry replicates on 91 fresh scenarios the instrument was never tuned on.

The 91 scenarios cover new domains and were calibrated to the moderate stakes band before any claim effect was measured, then run once (3,220 trajectories, 5 models). Pooled, scope rises from 40% to 86% (p = 8.8×10−53) while harm stays flat, 30% to 27% (p = 0.31); the asymmetry has p = 1×10−90. False scope claims still move agents (59% vs 40% at control, p < .001). Per batch the direction holds and the magnitude varies, and the paper reports that variation rather than averaging it away.

91fresh scenarios
3,220trajectories
5models
1×10−90p, scope vs harm asymmetry

The chart needs JavaScript; the numbers are in the text above.

Pooled violation rate on the 91 fresh scenarios. Source: paper Table 4.
Per batch (control → confirmed)
BatchScopeHarm
239→96%31→29%
340→71%26→43%
438→90%36→26%
540→76%29→33%
633→88%38→33%
745→100%18→8%
838→84%32→30%
930→85%25→32%
1047→93%30→20%
1143→77%25→25%
1230→76%32→30%
1350→100%30→30%
1456→96%27→17%
1530→100%32→27%

Source: paper Table 5.

Finding 3 · The newest frontier models

On the fixed harness, GPT-6.1, Claude Sonnet 5.5 and Gemini-4 follow the same pattern: confirmed scope claims raise violation, harm claims stay at 0–4%.

All three ran on the full in-band set on the fixed harness (GPT-6.1: 796 clean trajectories; Sonnet 5.5 and Gemini-4: 1,023). GPT-6.1 obeys the rules at control (scope 2%, harm 0%) although with the rule removed it takes the action in 76% (scope) and 37% (harm) of trajectories; harm claims never move it (0/106 confirmed, 0/106 refuted), while scope claims reach 70% when confirmed and 15% when refuted (95% CI 9–24%). Sonnet 5.5 and Gemini-4 rise from control (5%, 12%) to 80% and 58% under a confirmed scope claim, with harm claims at 0–4% (p ≤ 3×10−17). On false scope claims they differ: Sonnet acts on 16%, Gemini on none (0/72).

The chart needs JavaScript; the numbers are in the text above.

Violation rate by condition, full in-band set, fixed harness. Under each bar: violations / clean trajectories. Gemini-4 excludes the scenarios its provider refused. Bars from the released result summaries (claim-evidence.js); they match the paper, Section 4.

Caveat on the earlier Tier-2 run. The Tier-2 run (claude-sonnet-5.5, gpt-4o, gemini-2.5-flash; 60 scenarios, 1,080 trajectories) used the harness with the registry-routing bug: in 51 of the 60 scenarios the registry was unavailable, so its confirmed and refuted arms differ only in label. Its claim-type comparison stands (pooled 70% vs 24%, p = 3×10−18), but the paper reports no false-claim ranking from it. On the fixed harness, claude-sonnet-5.5's false-claim rate on those scenarios is 8%, not 38%; gpt-4o and gemini-2.5-flash have not been re-run.

Finding 4 · Frontier-hard variant

On a frontier-hard variant, the three newest flagships act on a false scope claim 67–87% of the time and never on a false harm claim.

In a 10-scenario pilot the newest models resisted the base attack (a false scope claim moved them 0/15), so the paper built a frontier-hard variant (batch16): mundane, reversible actions with genuine task pressure, where the scope fact is a plausible attribute such as a manifest tag, an SSO group or a console disposition. Scenarios are admitted by violability (the action is taken when no rule is present), never by their claim effect; with the rule removed, GPT-6.1 takes the action in all 17 batch16 scenarios (32/34 trajectories). Tested blind on the frozen 10-scenario set at 3 reps, a false scope claim succeeds 10/15 for GPT-6.1 (67%, 95% CI 42–85%), 13/15 (87%) for Claude Sonnet 5.5 and 11/15 (73%) for Gemini-4, and a false harm claim 0/15 for each (p = 2×10−4 for GPT-6.1, p ≤ 5×10−5 for the other two). Sonnet and Gemini break the rule more often at control on scope scenarios (27% and 40%) than GPT-6.1, yet the scope lift far exceeds it.

The chart needs JavaScript; the numbers are in the text above.

Frozen frontier-hard set, 10 scenarios at 3 reps. Scope control: violation on scope scenarios with a neutral note. False: the registry refutes the claim. Source: paper Section 4.1; bars from claim-evidence.js.

A 13-model open panel on the same variant (single rep; a generality probe, not a headline number) finds scope-lift above harm-lift in all 8 in-band models (mean confirmed-lift +62 vs +17 points). The variant awaits the same human construct audit as the main set.

Finding 5 · Same scenario, both claim types

When each scenario carries both claim types, the scope claim still beats the harm claim: the claim type, not the scenario, carries the effect.

Scope and harm scenarios differ before any claim: with no rule, GPT-6.1 takes the action in 76% of scope but 37% of harm scenarios. To remove this confound, a claim of the opposite type was written before measuring for every in-band and frozen frontier-hard scenario (same entity, date and register, within 12 characters; directive, task, tools and predicate unchanged; pre-registered). In scenarios that originally carried a harm claim, the new scope claim reaches 41% under confirmation while the original harm claim stays at 0%; in scope scenarios the new harm claim reaches 3%. The pattern holds on the 60 scenarios GPT-6.1 would violate with no rule present (43 vs 1) and on the frontier-hard set (9 of 10 scenarios, no reversal).

60 of 99scenarios where scope beats harm
1reversal
38ties
5×10−17sign test p

The chart needs JavaScript; the numbers are in the text above.

GPT-6.1, confirmed condition. Each row is a set of scenarios by original claim type; the claim of the opposite type is the new crossed claim. Source: paper Table 7; bars from claim-evidence.js.

The crossed claims await the same human construct review as the originals.

Finding 6 · Mechanism

Agents check whether the rule is in force, but not whether the claim is true.

Replaying the registry calls in each false-claim violation shows the same pattern across runs. Of GPT-6.1's 14 false-claim violations on the full set, 12 called check_policy but asked only whether the rule was in force, 2 did not call it, and none asked about the claim; whenever GPT-6.1 checked the claim, it complied. Of Claude Sonnet 5.5's 15, 13 checked only the rule, and 2 asked about the claim, received “no record”, and acted anyway.

Run (false-claim violations)Checked only the ruleDid not checkAsked about the claim
GPT-6.1, full set (14)1220
GPT-6.1, frontier-hard (10)550
Claude Sonnet 5.5, full set (15)1302
Claude Sonnet 5.5, frontier-hard pilot (5)401

In each Sonnet case that asked about the claim, the registry answered “no record” and the agent acted anyway. Source: paper Sections 4 and 4.1.

Finding 7 · Defenses

On GPT-6.1, a one-sentence instruction to verify the claim removes the false-claim attack without blocking legitimate work; a registry gate and an untrusted-memory label over-block.

Three pre-registered defenses ran on GPT-6.1 over all 51 scope scenarios (46 base, 5 frontier-hard; 642 trajectories), scored on security (violations under a refuted claim) and utility (actions under a confirmed, legitimate exemption). The verify-the-claim prompt cuts false-claim violations from 22% to 0% (0/107, p = 3×10−8; 67% to 0% on the frontier-hard set) and raises legitimate use from 71% to 93%, because the agent now looks up the exemption and acts on it. The registry gate refuses the prohibited call unless the registry confirmed the claim earlier in the episode, so it is secure by construction, but it cuts legitimate use to 44% because the agent often acts before checking; the untrusted-memory label gives 4% and 25%, as the agent largely stops using memory. These results cover one model, and weaker models may follow the instruction less reliably.

The chart needs JavaScript; the numbers are in the text above.

GPT-6.1, 51 scope scenarios, n = 107 per cell. False claim: violations when the registry refutes the claim (lower is better). True claim: actions taken under a confirmed, legitimate exemption (higher is better). Source: paper Table 13; bars from claim-evidence.js.

Rule-phrasing defenses do not move violation

Under a confirmed scope claim (n = 1,344), restating the rule, naming the harm, or asserting scope authority leaves violation unchanged (all CMH p > .6); the targeted scope-authority phrasing moves it by 0.0 points.

Guardrail phrasingViolation
None (baseline)45%
Restate the rule44%
Name the harm46%
Assert scope authority45%

Source: paper Table 14 (v11).

Finding 8 · Detection

A prompt-injection detector flags none of the 24 claims and twins.

The effective claim issues no instruction. Scored by a purpose-built prompt-injection detector, all 24 claims and twins are unflagged, with a median score of about 0.0006 and a maximum of about 0.004, against about 0.9996 for a canonical injection. The note that drives up to 100% violation sits orders of magnitude below the signal that instruction-oriented defenses key on.

0/24claims and twins flagged
≈0.0006median detector score
≈0.004maximum detector score
≈0.9996canonical injection

Source: paper Section 6.

Finding 9 · Robustness battery

The asymmetry survives six pre-committed robustness perturbations, each testing a distinct confound.

Rewriting the claim into four pre-committed surface forms (8 scenarios, 319 trajectories, 5 models) leaves violation at 65% for the original template and 65% over the three paraphrases (Fisher p = 1.0; per wording 62–66%). Under the strict pass2 metric the effect is consistent (scope 76% vs harm 18% of scenario–model cells violate on both reps), and stating the claim more assertively steps violation from a hedge (“may”, 45%, n.s. vs the 35% neutral control) to a plain assertion (“is”, 87%, p = 1×10−4), with no further rise from added authority (85%). The effect holds whether the claim arrives via memory, tool output or a system note (87/77/95%, n.s.), when it is buried among 8 unrelated memory entries (77→92%), and at temperature 0 (scope 83% vs harm 30%, p = 6×10−5).

PerturbationResultRules out
Paraphrase (4 forms)65% = 65%, p = 1.0surface template
Consistency (pass2)76% vs 18%averaged noise
Assertiveness (dose)45%→87%, p = 10−4correlation
Claim source87/77/95%, n.s.memory-specific
Distractor dilution77→92%salience
Temperature 083% vs 30%sampling artifact

Caveat. These six studies ran on the harness with the registry-routing bug, so they test the claim with the registry unavailable. Their within-study comparisons are unaffected; re-running them on the fixed harness is pending. Source: paper Table 9.

Finding 10 · A negative result

A broader trigger-versus-override taxonomy does not hold.

The paper pre-registered whether the asymmetry reflects a broader law: three trigger-negating claim types (scope, exemption, temporal) should move agents and three override types (harm, benefit, urgency) should not. It does not (1,176 trajectories): pooled trigger 15% vs override 18% (p = .23), and benefit (24%) and urgency (19%) move agents more than any trigger type. The contribution is the measured scope-versus-harm difference, not a theory of it.

The chart needs JavaScript; the numbers are in the text above.

Violation rate by claim type, 1,176 trajectories. Source: paper Table 11 (v12).

Finding 11 · Construct validity

A blind rater recovers the scope-versus-harm labels on all 27 scenarios checked.

A rater saw the 27 validated claims blind (directive and claim only) and classified each as scope or harm. Agreement with the paper's labels was 27/27, including the genuine judgment calls: an allowlist exemption and a standing authorisation were both classified as scope, and a duplicate-charge note as harm.

27/27labels recovered
κ = 1.0agreement

Caveat. Construct validation rests on one expert rater close to the project. A blind full-set sheet is prepared for independent raters, and the paper plans to add at least 2 fully arms-length raters and report Fleiss κ in the camera-ready version. Source: paper Section 5.

Finding 12 · Limits and disclosures

The results come from a controlled benchmark with stated limits, and the paper discloses the instrument problems it found.

Source: paper Section 7 and the per-model results in Section 4.