ClaimHijackBench

ClaimHijackBench

Hijacking LLM Agents with a Single Benign Claim in Memory

Anonymous Authors

Under double-blind review

TL;DR

An agent must keep a rule. It then reads one harmless-looking note in its memory. If the note says the case falls outside the rule, the agent often breaks the rule, even when a trusted registry says the note is false. If a matched note says the harm will not happen, the agent keeps the rule. Agents check whether a rule is in force. They do not check whether the claim is true. One added sentence that tells the agent to check the claim stops the attack on GPT-6.1.

Three panels. (a) A matched harm claim in memory and the agent keeps the rule. (b) A false scope claim in memory and the agent issues the refund. (c) The agent checks the claim with the registry, gets no such record and refuses.
Figure 1. An example of how a scope claim in memory, but not a matched harm claim, can make an agent break a standing rule, and how checking the claim with the registry avoids it.

Abstract

LLM agents now act on real systems under rules written in plain language, such as guardrail files, system prompts and notes saved in memory. We show that one harmless-looking note in memory can make an agent break a rule it knows is in force. The note gives no order. We built ClaimHijackBench to test whether the kind of claim matters and whether agents check a claim before they act on it. It has 120 scenarios from 99 domains. Some notes say that a case falls outside the rule (a scope claim). Others say that the harm the rule guards against will not happen (a harm claim). A policy registry then confirms the note or says it is false. A violation is a logged forbidden tool call. Across 11 open and 5 frontier models, a confirmed scope claim makes agents break the rule in 86% of runs, while a matched harm claim stays at baseline. Scope claims still work 59% of the time when the registry says they are false. The three newest frontier models act on a false scope claim in 67–87% of hard trials and never on a false harm claim. The result holds on 91 new scenarios, and a prompt-injection detector flags none of the notes. One added sentence that tells the agent to check the claim cuts false-claim violations on GPT-6.1 from 22% to 0% without blocking real work. Agents check whether a rule is in force, but not whether the claim is true.

Overview in three steps: build a scenario, run the agent under a registry condition, score the logged tool calls with no LLM judge.
Overview of ClaimHijackBench. Each scenario has one rule, one memory note, a task whose direct solution breaks the rule, and tools with a registry. The agent runs under a registry condition that confirms or rejects the note. A violation is a forbidden tool call found in the log, with no LLM judge.

Citation

@misc{anonymous2026claimhijackbench,
  title  = {ClaimHijackBench: Hijacking LLM Agents with a Single Benign
            Claim in Memory},
  author = {Anonymous},
  year   = {2026},
  note   = {Under review}
}