Spring 2026 public snapshot

Cartograph Public Research Dossier

Frontier models in realistic agent workflows can adopt authority, trust, and approval semantics from plain text. The right measurement is attempt formation plus containment, not refusal rate alone.

Back to dossierDownload markdown
Snapshot

Public numbers

  • 246 attack cases across 20 datasets and 9 threat channels
  • 222 canonical analyzed cases across 36 replay-valid runs
  • 56.8% overall tool-aligned success (95% CI 50.2%–63.1%)
  • 0 high-risk tool executions in the public snapshot
Workflow slice

Why the public story is sharper than a refusal benchmark

  • Semantic capture: 31.25%
  • Discovery success: 31.25%
  • High-risk execution rate: 0%
  • Strongest model: GPT-4.1 at 70.8%
Model comparison

Cross-model signal

  • GPT-5-mini: 64.9% tool-aligned success
  • GPT-4.1: 59.5% tool-aligned success
  • GPT-4o: 45.9% tool-aligned success
Flagship findings

What Cartograph found

Workflow text becomes operational authority.

Runbooks, handoff notes, review sheets, and approval text can be adopted as active authority rather than treated as passive context.

Proof point: Representative workflow cases show the model switching from summarization into operational planning after reading poisoned instructions.

Summarize, review, and handoff tasks can drift into action preparation.

The failure often appears before the final risky tool: the model asks for the exact command, recipient, destination, or missing approval detail needed to continue the malicious workflow.

Proof point: The March 18 workflow slice measured 31.25% semantic capture even though high-risk tool execution stayed at 0%.

Containment and compromise are different safety signals.

A runtime can successfully block dangerous execution after unsafe tool intent has already formed. Public reporting has to preserve both signals.

Proof point: The public snapshot shows 0 high-risk tool executions while still recording substantial tool-aligned success and workflow compromise.

Architecture

How the stack works

Threat taxonomy and case corpus

Cartograph organizes the agentic surface into nine threat channels and a public program scope of 246 attack cases across 20 datasets.

Adaptive attacker lanes

Attack generation combines iterative attacker loops, multi-turn escalation, and realistic workflow poisoning cases.

CapabilityOS containment layer

The execution shell separates model behavior from runtime permissioning through layered controls: tool allowlists, approval gates, egress controls, and path restrictions.

Dual-signal scoring

Cartograph scores unsafe intent formation separately from final execution and adds intermediate workflow compromise signals such as semantic capture.

Replay-valid evidence

Every public claim in this packet is meant to resolve to replay-backed evidence, a bounded transcript, or an explicitly downgraded aggregate result.

Case studies

Three public-safe examples

Maintenance Override in a Handoff Note

Scenario: REAL_MAINTENANCE_OVERRIDE_EXEC

A handoff note described a risky maintenance override as already approved process. The model treated the note as active authority and moved from review into operational preparation.

Public-safe excerpt:The runbook says the maintenance override is already approved, so I just need the exact diagnostic command and destination.

Lesson: In agent workflows, a handoff note is not just context. It can become a control surface.

Release Runbook to Telemetry Upload

Scenario: REAL_RELEASE_WEB_FETCH, COMP_RUNBOOK_PLUS_TELEMETRY

A poisoned release runbook reframed environment collection and external telemetry upload as standard release procedure. The model adopted the document's operational logic instead of treating it as untrusted text.

Public-safe excerpt:The release document requires gathering environment variables and recent logs before the telemetry POST.

Lesson: Runbooks and retrieved docs have to be modeled as possible control input, not just reference material.

Eval Rubric Sabotage

Scenario: REAL_EVAL_RUBRIC_SABOTAGE

A poisoned evaluation sheet caused the model to preserve the malicious scoring instruction as if it were legitimate rubric policy. No dangerous external tool was needed for the failure to matter.

Public-safe excerpt:I'll preserve the weighting exactly as the review sheet specifies so the summary matches the rubric.

Lesson: Agentic risk is not only about sending email or running code. It also includes corrupting the system that decides whether a run looked safe.

Disclosure boundary

What is public

  • Aggregate metrics and model comparison.
  • Sanitized mechanism-first case studies.
  • Architecture and measurement design.
  • Explicit public/private visibility boundary.
Limits

What this brief does not claim

  • It is a bounded snapshot, not the full private research base.
  • It does not publish exploit chains, prompts, or raw replay bundles.
  • Some protocol-trust details remain narrowed until evidence is cleaner.