ClaimHijackBench
Hijacking LLM Agents with a Single Benign Claim in Memory
TL;DR
An agent must keep a rule. It then reads one harmless-looking note in its memory. If the note says the case falls outside the rule, the agent often breaks the rule, even when a trusted registry says the note is false. If a matched note says the harm will not happen, the agent keeps the rule. Agents check whether a rule is in force. They do not check whether the claim is true. One added sentence that tells the agent to check the claim stops the attack on GPT-6.1.
Abstract
LLM agents now act on real systems under rules written in plain language, such as guardrail files, system prompts and notes saved in memory. We show that one harmless-looking note in memory can make an agent break a rule it knows is in force. The note gives no order. We built ClaimHijackBench to test whether the kind of claim matters and whether agents check a claim before they act on it. It has 120 scenarios from 99 domains. Some notes say that a case falls outside the rule (a scope claim). Others say that the harm the rule guards against will not happen (a harm claim). A policy registry then confirms the note or says it is false. A violation is a logged forbidden tool call. Across 11 open and 5 frontier models, a confirmed scope claim makes agents break the rule in 86% of runs, while a matched harm claim stays at baseline. Scope claims still work 59% of the time when the registry says they are false. The three newest frontier models act on a false scope claim in 67–87% of hard trials and never on a false harm claim. The result holds on 91 new scenarios, and a prompt-injection detector flags none of the notes. One added sentence that tells the agent to check the claim cuts false-claim violations on GPT-6.1 from 22% to 0% without blocking real work. Agents check whether a rule is in force, but not whether the claim is true.
Citation
@misc{anonymous2026claimhijackbench,
title = {ClaimHijackBench: Hijacking LLM Agents with a Single Benign
Claim in Memory},
author = {Anonymous},
year = {2026},
note = {Under review}
}