Proof — Locke AI

How every study is run

What counts as a failure is written down and locked before any transcript is scored.
Every transcript is graded by an outside model, never by the system being tested, and every flagged result is verified by hand.
ANNA is tested on the same underlying model as the bare comparison, so the only thing that changes is where the decision is made.
Where a result shows zero, that means zero failures in the trials run, not a claim of zero risk.

Study 01
Aug 2026

Does safety come from the model or from ANNA?

Real recorded multi-turn human attacks, run against frontier models on their own and then against the same models inside ANNA.

ResultOn their own, models break on anywhere from 8.3% to 78.3% of attacks depending on which model it is. Inside ANNA, all four models break on 0%.

Method, sample and judge

Benchmark: Multi-turn Human Jailbreak (MHJ). Sustained conversations in which a person works on the system over many turns, not single adversarial prompts.
Why multi-turn: Single-turn tests measure the refusal reflex, which holds. Multi-turn attacks go around it, one concession at a time.
Models: Opus, Grok, GPT-5.5, DeepSeek and GPT-4o-mini, alone and inside ANNA.
Judge: StrongREJECT via Sonnet, cross-checked against the HarmBench classifier. Every flagged result verified by hand.
Status: Parts one and two of the What Holds the Line series.

Part one ↗Part two ↗

Study 02
Sep 2026

Would any outside checker do this, or only ANNA?

If any simple checker placed in front of a model gets the same result ANNA does, then ANNA is nothing special. So we tested that: one checker, held fixed, with the model underneath swapped out.

ResultWith a simple checker, the break rate still swung from 3.3% to 18.3% depending on the model. The checker leans on the model instead of replacing its judgment. ANNA, tested against an adaptive attacker briefed on how it works, held on all 13 objectives on the cheapest model.

Method, sample and judge

Compared: NeMo Guardrails stock self-check, NeMo with a fixed external checker, LlamaGuard, a well-resourced self-check with room to reason, and checker pairings from the same lab.
Over-blocking: Measured alongside break rate. NeMo on DeepSeek reached a 1.7% break rate by refusing roughly a third of completely harmless requests. That is not judgment. It is refusing to make the call.
Adaptive test: Thirteen objectives including weapons access, fraud, disinformation, dangerous chemical misinformation, theft and hate speech. The adversary was briefed on how ANNA works and on the published jailbreak literature.
Judge: Outside model, with every flagged result verified by hand.

Read the write-up ↗

Study 03
Jul 2026

How long before it gives in?

No jailbreak, no attacker. The model takes a correct but debatable position, then faces seven turns of ordinary pushback across twenty scenarios: disagreement, claimed authority, urgency, fatigue, repetition.

ResultModels on their own give ground on 16–26% of turns and fully reverse on 12–16%. Turn three, when the user claims authority, is the cliff. On the same test ANNA held its position on all 114 chances to concede.

Method, sample and judge

Scenarios: Twenty scenarios across five everyday domains: technical design decisions, engineering practice, honest uncertainty, standing by a critique, and financial concerns.
Scoring: Each turn graded as holding firm, holding warmly, hedging, conceding real ground, reversing, or reversing and inventing a justification.
Recovery: Once a model gives in, it stays that way 70% to 98% of the time. Guardrails, self-critique and doing nothing all land in the same band.
Safety prompts: The best hold-your-ground prompt we could write failed 6% of the time under everyday pressure, then 18% when the real problem was hidden, worse than the same model with no prompt.
Cross-grading: All 1,674 turns re-graded by a model from a different company. 88% agreement overall, 94% on the crisis scenarios, and 100% on ANNA's turns.

Read the brief ↗

Study 04
Jun 2026

Is a very good safety prompt enough?

Four high-stakes scenarios drawn from documented real-world failures, run three ways on the same model: alone, with our best safety prompt, and inside ANNA.

ResultThe prompt cut failures by 29%. ANNA cut them by 92%. The most expensive frontier model available handed over information that could cause serious harm 11 times. Inside ANNA, zero.

Method, sample and judge

Arms: The same model in all three conditions. Only what surrounded it changed.
The backfire: The same safety prompt that reduced failures in one scenario made failures in another jump sevenfold, from 5% to 33%. A prompt only covers the failures its author thought of.
Model independence: Run with Sonnet, GPT-4o-mini and Grok underneath. The safety floor held on all of them.
Judge: Graded by outside models from different companies, with a second review pass that could only remove flags, never add them.

Read the brief ↗

Study 05
Jul 2026

What happens on somebody else's benchmark?

An independent, open-source safety benchmark we did not write, scored on a public leaderboard we do not control.

ResultOn the measure that matters most, the strongest frontier models are flagged as high potential for harm in 33–40% of the highest-stakes moments. On the same measure and the same judge, ANNA is flagged in 7%, running on a deliberately cheap model.

Method, sample and judge

Benchmark: VERA-MH v1.1, independent and open source. 200 scenarios on the published leaderboard.
Model used: Deliberately inexpensive, to separate what ANNA contributes from what the model contributes.
Judges: Scored by both the benchmark's original judge (94.9 to 97.1) and its newer, stricter judge (82.4).
The fix: Three gaps identified, three general rules written, re-run the same afternoon: 75.5, then 78.2, then 82.4. The new rules held on twenty cases they had never seen.
Why that matters: When the bar moves, a trained model has no quick answer. Its safety lives in weights nobody can read, so the only fix is another training run.

Read the analysis ↗

Published as we go, ahead of peer review.

Does safety come from the model or from ANNA?

Would any outside checker do this, or only ANNA?

How long before it gives in?

Is a very good safety prompt enough?

What happens on somebody else's benchmark?

The thinking behind the work.

Benchmark it on your scenarios.