ClaimHijackBench
Hijacking LLM Agents with a Single Benign Claim in Memory
TL;DR
ClaimHijackBench gives an agent a rule and one harmless-looking note in its memory. Across 120 scenarios and 16 models, a note that says the case falls outside the rule makes agents break it in 86% of runs, while a matched note that says the harm will not happen stays at baseline. The three newest frontier models act on a false scope note in 67–87% of hard trials.
Abstract
LLM agents now act on real systems under rules written in plain language, such as guardrail files, system prompts and notes saved in memory. We show that one harmless-looking note in memory can make an agent break a rule it knows is in force. The note gives no order. We built ClaimHijackBench to test whether the kind of claim matters and whether agents check a claim before they act on it. It has 120 scenarios from 99 domains. Some notes say that a case falls outside the rule (a scope claim). Others say that the harm the rule guards against will not happen (a harm claim). A policy registry then confirms the note or says it is false. A violation is a logged forbidden tool call. Across 11 open and 5 frontier models, a confirmed scope claim makes agents break the rule in 86% of runs, while a matched harm claim stays at baseline. Scope claims still work 59% of the time when the registry says they are false. The three newest frontier models act on a false scope claim in 67–87% of hard trials and never on a false harm claim. The result holds on 91 new scenarios, and a prompt-injection detector flags none of the notes. One added sentence that tells the agent to check the claim cuts false-claim violations on GPT-6.1 from 22% to 0% without blocking real work. Agents check whether a rule is in force, but not whether the claim is true.
Citation
@misc{anonymous2026claimhijackbench,
title = {ClaimHijackBench: Hijacking LLM Agents with a Single Benign
Claim in Memory},
author = {Anonymous},
year = {2026},
note = {Under review}
}