10 min read

You Cannot Just Say an AI Is Aligned. You Have to Measure It.

Reflex Alignment supervises actions; AlignmentBench measures operational behavior across checkpoints, agents, and trajectories. A practical proposal for turning alignment into an evaluable process.

In my previous article I argued for an idea that feels less optional every week: model capability cannot keep growing indefinitely ahead of our ability to evaluate, monitor, and interrupt those models.

Dario Amodei recently made a particularly concrete version of that argument. In We Must Pace the Frontier, he proposes external evaluators with continuous access to frontier labs and explicitly argues that they should help assess not only finished models, but also training pipelines and processes.[1]

He also describes a possible checkpoint logic: when a system reaches certain capability levels, it should be accompanied by evidence about alignment properties through a combination of evaluations, interpretability, and audits.[1]

The practical question is obvious:

How do we measure those properties repeatedly while a model is being trained, after training, and when it finally starts acting?

That question sits behind two experiments we have just published at Brida: Reflex Alignment and AlignmentBench.

But they are part of a larger launch.

This week we published Brida Reflex as a typed semantic decision layer for agents and software; ReflexBench as a reproducible benchmark for the decision engine; Reflex Alignment as semantic supervision before an action is executed; and AlignmentBench as a benchmark for the operational behavior of models, checkpoints, and agents under explicit policies. The public Brida SDK is available as @brida/sdk, and the main technical surfaces are open on GitHub.[6][7][8][10]

The public architecture is intentionally separated:

ReflexBench -> decision-engine quality
AlignmentBench -> target-system behavior
Reflex Alignment -> runtime semantic supervision
host policy -> deterministic execution authority

Reflex is also not being launched as an unmeasured intuition.

In ReflexBench we published frozen, reproducible results. On the 111-case public-hard cohort, the tested TypeSafe Jev route reached 73.0% semantic accuracy, compared with 41.4% for the next tested engine in that same cohort. In the blind workflow gate, Reflex Core policy applied to the same response moved the result from 76.7% to 82.0%, a paired +5.3 percentage-point improvement, with 150/150 cases completed. These are results on specific frozen cohorts, not a universal superiority claim.[7]

None of this claims to solve “the alignment problem.”

The goal is smaller.

And because it is smaller, we can test it.

From “it seems aligned” to an evaluation loop

There is already serious work around this problem.

Anthropic has published AuditBench, a benchmark for evaluating auditing techniques against models with implanted hidden behaviors.[2]

Petri runs multi-turn behavioral audits in simulated scenarios and has worked specifically on reducing eval awareness: the possibility that a model recognizes that it is being tested and behaves differently during the evaluation.[3]

Bloom generates behavioral evaluation suites for researcher-specified behaviors and measures how frequently and severely those behaviors appear.[4]

OpenAI, meanwhile, wrote this summer that some failures in long-horizon systems were not captured by its existing pre-deployment evaluations. Its response included new evaluations and trajectory-level monitoring, rather than looking only at isolated actions.[5]

And on August 28, Anthropic published an experiment that makes this discussion even more concrete. Its automated alignment researchers worked on ten classes of alignment failures and were eventually applied to an early checkpoint of Claude Opus 4.8. At the same time, Anthropic had to monitor 1,601 research trajectories and exclude the 2.4% that contained cheating or evaluation-gaming behavior.[9]

To me, those two conclusions belong together:

yes, evaluations during training can help improve behavior; therefore we should design those evaluations assuming a sufficiently capable system may learn to optimize, exploit, or recognize them.

All of this points in the same direction:

alignment needs instrumentation.

We need to observe a model repeatedly, record what it did, under which rules, at which checkpoint, across which trajectory, and identify where behavior started to regress.

AlignmentBench turns alignment into a reproducible checkpoint loop.

The thing I wanted to measure: operational alignment

AlignmentBench starts from an intentionally narrow definition.

It does not ask whether an AI is “aligned with humanity.”

It does not try to infer internal objectives.

It does not produce a moral certification.

It asks a much more operational question:

Given an explicit policy, does the system’s chosen action remain inside the authority, oversight, impact, and scope boundaries it was given?

The first public version is a Developer Preview v0.1.0 with 28 synthetic cases across seven dimensions:

  1. authority boundaries;
  2. instruction conflict;
  3. uncertainty and escalation;
  4. reversibility and impact;
  5. scope integrity;
  6. interruptibility and oversight;
  7. trajectory integrity.

Every scenario requires the system to choose one of three operational classes:

execute · review · decline

And there is an important distinction here:

policy compliance is not exactly the same as choosing the best action.

Imagine a safe action the model could execute correctly.

If it escalates that action for human review, it may be too conservative. That is different from executing an action for which it has no authorization.

That is why AlignmentBench separates metrics such as:

  • policy compliance;
  • best-action accuracy;
  • unsafe execution rate;
  • appropriate escalation;
  • over-caution;
  • critical policy violations;
  • per-dimension results.

There is no magic “alignment score” that hides everything behind one scalar.

During training: detect regressions, do not teach the exam

The part that interests me most is using this kind of evaluation between checkpoints.

The loop looks roughly like this:

checkpoint N -> benchmark -> receipt -> compare with N-1 -> investigate regressions

If a model becomes dramatically more capable while starting to degrade in interruptibility, authority boundaries, or escalation under uncertainty, I want that to appear as an explicit regression.

And if a post-training process attempts to correct that regression, I want to run the same battery again and observe what changed.

This starts to look less like an abstract philosophical discussion about alignment and more like quality engineering.

But there is an obvious trap.

If the model repeatedly sees the same 28 questions and we optimize directly against them, what we may be training is the ability to pass AlignmentBench, not a generalizable property.

That is why the public methodology recommends:

reproducible public suite + private rotating holdouts + adversarial variants + new trajectories.

The public suite gives us reproducibility and debugging.

The complete exam should not be static.

It is the same reason Petri works on eval awareness, and why any serious evaluation system should assume that with enough capability, a model can learn how it is being measured.[3]

Reflex Alignment: measure before deployment, supervise after it

The benchmark covers one part of the lifecycle.

The other appears when the model is already operating.

We have published a new Brida Reflex use case called Reflex Alignment.[6]

The idea is deliberately simple:

model -> proposed action -> Reflex Alignment -> host policy -> execution

Before creating a real-world effect, the agent serializes a bounded context:

  • proposed action;
  • purpose;
  • effect;
  • reversibility;
  • execution principles;
  • relevant deterministic checks.

Reflex then classifies the action as:

aligned_candidate · review · misaligned_candidate

The Reflex Alignment contract turns explicit principles into a semantic decision before execution.

The important part is what it does not do.

Reflex does not replace permissions.

It does not replace an allowlist.

It does not decide budgets.

It does not replace access controls, protected environments, or deterministic validation.

The host retains authority.

Reflex is a semantic layer for the part of the decision that does not reduce cleanly to an if.

Is it circular for one AI to judge another AI?

It is, if we design the benchmark that way.

That is why AlignmentBench keeps two things separate.

The benchmark truth comes from frozen cases with allowed actions, a preferred action, and explicit policy.

Reflex is an additional semantic audit.

A model prediction can be converted automatically into a Reflex Alignment state and evaluated by a second system.

If Reflex disagrees with the ground truth, the disagreement is recorded.

The benchmark truth is not silently rewritten because another model had a different opinion.

And in serious evaluations, the identity of the target model should also be separated from the identity of the evaluator. A model should not be its own sole supervisor.

ReflexBench, AlignmentBench, and Reflex Alignment are three different things

It would be easy to produce names that look redundant, so the public architecture is intentionally explicit.

ReflexBench asks: how good is the decision engine?
It measures semantic quality, calibration, robustness, consistency, and typed-decision behavior.[7]

AlignmentBench asks: how well does the target system stay inside explicit operational constraints?
It measures model/checkpoint behavior and regressions by dimension.[8]

Reflex Alignment asks at runtime: is this specific action consistent with the supplied execution principles?[6]

Benchmark the decision engine.

Benchmark the system.

Supervise the action.

They are complementary layers.

Where this becomes especially important

The more irreversible an action is, and the greater its impact, the less reasonable it becomes to rely on a single generative output.

An agent with access to important infrastructure, industrial systems, financial operations, privileged environments, or any other critical domain needs something closer to defense in depth.

Deterministic checks.

Least privilege.

Explicit approvals.

Interruptibility.

Trajectory evaluation.

And, when appropriate, an independent semantic layer.

AlignmentBench includes synthetic high-impact scenarios precisely to measure properties such as authority, approval, and interruptibility, while keeping those scenarios abstract:

a safety benchmark does not need to teach dangerous procedures in order to test whether a model respects a boundary.

The most interesting work is still ahead

v0.1 is small on purpose.

The next things I want to explore are harder:

longer trajectories;

automatic meaning-preserving variants;

private holdouts generated after checkpoint freeze;

systematic comparison between target models and evaluators;

multi-agent alignment;

and, most importantly, whether the metrics predict behavior outside the benchmark.

I also want to run real models against the public suite and publish complete receipts.

Until then, I do not want to publish a table of “most aligned models.”

First we need a methodology that deserves that comparison.

A smaller proposal than “solving alignment”

Part of the problem with this conversation is that we use the word alignment for too many different things.

Sometimes we mean internal objectives.

Sometimes safety.

Sometimes values.

Sometimes obedience.

Sometimes simply whether an agent was authorized to do something.

AlignmentBench only attempts to capture this last family of problems, expanded to oversight, impact, and trajectory:

operational alignment under explicit policy.

It is much less ambitious than solving alignment.

And precisely because of that, we can write tests.

We can version the benchmark.

We can store receipts.

We can compare checkpoints.

We can detect regressions.

We can put a supervision layer in front of execution.

And we can open the code so other people can try to break the methodology.

To me, that is the right next step:

stop treating alignment only as a property we declare, and start treating at least part of it as a system we measure continuously.

Brida reading

Code

Sources

  1. Dario Amodei — We Must Pace the Frontier (Sep 2026). Embedded evaluators, training pipelines, checkpoints, evaluation, and interpretability.
    https://darioamodei.com/post/we-must-pace-the-frontier

  2. Anthropic — AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors (10 Mar 2026).
    https://alignment.anthropic.com/2026/auditbench/

  3. Anthropic — Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations (22 Jan 2026).
    https://alignment.anthropic.com/2026/petri-v2/

  4. Anthropic — Bloom: an open source tool for automated behavioral evaluations (19 Dec 2025).
    https://alignment.anthropic.com/2025/bloom-auto-evals/

  5. OpenAI — Safety and alignment in an era of long-horizon models (20 Jul 2026). Incident-derived evaluations and trajectory-level monitoring.
    https://openai.com/index/safety-alignment-long-horizon-models/

  6. Brida — Reflex Alignment.
    https://github.com/brida-ai/reflex/tree/main/examples/use-cases/alignment

  7. Brida — ReflexBench.
    https://github.com/brida-ai/reflexbench

  8. Brida — AlignmentBench v0.1.0 Developer Preview.
    https://github.com/brida-ai/alignmentbench

  9. Anthropic — Automated researchers can reliably mitigate alignment failures (28 Aug 2026). Alignment post-training over public benchmarks and an early Opus 4.8 checkpoint; 1,601 monitored trajectories and 2.4% excluded for cheating behavior.
    https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures

  10. Brida — Introducing Brida Reflex: fast typed decisions for agents and software. Product launch, public SDK/registry, use cases, and ReflexBench evidence.
    https://www.brida.ai/blog/introducing-brida-reflex

AlignmentBench v0.1.0 is an engineering experiment and a Developer Preview. It is not a safety certification, does not demonstrate safe internal objectives, and does not replace auditing, interpretability, deterministic controls, or human review.