Measuring exposure detection under hard negatives

LS-Stat is the detector inside Interocept: it reads an agent model's internal activations and scores whether the context it's about to act on contains injected content. Before we'd put a claim like "detects injection exposure" in front of a customer, we set ourselves a gate for it — and it's worth saying plainly that the gate is not met yet. This is the write-up of what we measured, what we tried, and what that means for how we describe the product today.

What the gate measures

The label LS-Stat learns is exposure, not compliance: positive means injected content was present in the context window, regardless of whether the agent obeyed it. That distinction matters, because it means the hardest negative examples aren't obviously-clean traffic — they're content that looks like an instruction but isn't a live directive. A closed incident report quoting an old attack. A security advisory. A ticket description.

Our target, set before we measured anything: true positive rate ≥ 0.80 at a false positive rate ≤ 0.01, on a corpus mixing clean sessions with these hard negatives. On our staged AgentDojo + InjecAgent corpus with a held-out attack family, the shipped detector reaches 0.1237 — a wide miss, and one that gets worse (0.0825) once we widen the corpus to 25 attack families instead of 7.

The headline number moves more than the model does

Before assuming the fix has to be a better model, it's worth showing how much the number above depends on something that has nothing to do with the detector: what counts as a "negative" in the eval set.

Scoring the exact same probe, on the exact same held-out split, against different slices of negative traffic:

Nothing about the model changed across that table — only which traffic counted as a negative. That's a real finding on its own: a single global gate number, without naming the traffic it's scored against, isn't a stable target to design a product around. It's also not a way out — even the most favorable slice above (0.72) still falls short of 0.80.

Where the detector actually struggles

Per-slice, the picture is more specific than the headline number suggests. LS-Stat is a strong signal for the core product question — an injected instruction versus the user's own authorized one — beating a character n-gram baseline by 0.11 AUROC there, and by 0.14 on clean twins.

It's a weaker signal for one thing specifically: telling live instructions apart from the same text, reproduced as inert evidence. On that slice alone, the shipped detector trails the n-gram baseline by 0.14 AUROC (0.65 vs. 0.79) — worse than a naive baseline, on the exact slice a security team ingesting its own incident reports would generate constantly. Given that slice is roughly a sixth of the negatives in our standard mixture, it's most of why the headline number is what it is.

Why not just a regex?

Fair question, and on raw in-distribution accuracy the honest answer is "you might not need one yet." A plain character n-gram classifier essentially ties the activation probe overall (0.9502 AUROC against 0.9523), and beats it outright on two of the slices the product needs to win on — quoted-inert content and other in-domain negatives. If the only thing that mattered were accuracy on the exact corpus we trained on, a text classifier would be the simpler, cheaper answer, and we'd rather say that plainly than let a comparison we didn't run go unaddressed.

Two places the activation channel earns its cost, measured against the same n-gram baseline:

Generalizing to an attack family neither model has seen, mean AUROC / TPR @ 1% FPR across four corpora:

The probe leads on AUROC across all four and on TPR for both short-session corpora; the n-gram baseline pulls ahead on TPR for both long-session corpora. We're not rounding that off — it's a real, uneven result.

Holding up against an attacker who rewrites the payload once they know they're being scored — recall at a threshold fixed for 1% FPR on clean traffic, before and after rewriting:

On short sessions the probe holds its recall under paraphrasing entirely, where the n-gram baseline loses over a third of it. On long sessions the picture is more mixed, and the n-gram baseline starts from a higher number in absolute terms either way.

That's the actual case for reading activations instead of text: not that it's more accurate on the data we already have, but that it degrades more gracefully against an attack family it hasn't seen and an attacker who knows it's there. Whether that trade is worth the added infrastructure is a question we'd rather hand a customer both numbers for than answer on their behalf.

Four attempts to close the gap

We treated closing this as an open question rather than a tuning exercise, and tried four different levers. Reporting all four here, not just the ones that worked — a negative result is still real information about where the signal does and doesn't live.

1. An additive channel for live-vs-reported content. When the operative/reported distinction is isolated on its own, the signal is genuinely there — a probe trained just on that axis reaches 0.71 AUROC where a char n-gram sits at chance. Adding it as a second feature to the primary detector's score didn't transfer that: +0.002 on the quoted-inert slice, and every other slice fell. The signal exists; naive fusion destroyed it rather than combining with it.

2. A routing architecture instead of fusion. If addition doesn't work, maybe the two regimes need genuinely separate decision boundaries rather than one blended score. We built and tested this. The variant that routed between a specialized reported-content check and the primary detector's score performed worse than the primary detector alone — recall at the gate's tight operating point collapsed rather than improved. A version that let the data decide how much weight to give the routing signal did what a sensible fit does when a feature doesn't help: it assigned it close to zero weight and left the primary detector's score effectively unchanged.

3. A different backbone. Everything above used the same 3B-parameter decoder model reading its own activations. Substituting a much smaller bidirectional text encoder as the detector — architecturally suited to attending over a whole span at once rather than one direction through it — produced the most promising internal result so far: recall at the gate's operating point roughly doubled on the same held-out split, and the quoted-inert slice moved from below the character n-gram baseline to above it. This is still a preliminary, single-run result pending a clean held-out-family layer selection pass before we'd treat it as settled, and it still falls well short of the 0.80 target on its own — but it's the first lever this session that made the number move in the right direction by a meaningful amount rather than trading one slice for another.

4. Larger decoder backbones. In progress as of this write-up — evaluating whether a larger model in the same activation-probing family, with its own independently-selected layer band, closes any more of the gap. No result yet; we'll update this page rather than guess at one.

What we tell customers in the meantime

None of the above changes the honest claim today. Given where the numbers actually land, the defensible framing is high-recall detection of injection exposure, with deterministic fail-closed enforcement and a tamper-evident audit trail — not an autonomous gate that we'd ask a customer to trust blocking traffic on its own, yet.

That's not a hedge without a number behind it. On executed sessions, the tradeoff between blocking bad behavior and disrupting good work looks like this:

There's no operating point in that range where autonomous blocking is comfortable on today's evidence. So Latch, the enforcement layer, is built so every risk tier can independently run in escalate (hold for review, audited either way) or deny (block automatically) — and which mode is right is a decision we make per deployment, per tier, calibrated on a customer's own traffic, not a default we'd want to make for them in advance.

What would actually settle this

Not more analysis of the corpus we already have. Two things would move the needle in a way that isn't just re-slicing the same data: a customer's real negative traffic — the sensitivity table above shows how much that alone can shift the number — and enough calibration sessions (low hundreds, not tens) to certify a threshold with any confidence. Both only come from a real deployment, and building toward one honestly is a big part of why this page exists.