Testing Opus 5.5 as the code-review baseline

TL;DR

I would evaluate one Opus 5.5 reviewer at explicit effort before building a larger fleet. The baseline is a candidate for the next controlled comparison, with a qualified corpus and independent acceptance, not a model winner established by the earlier pilot.

Decision	Starting point for Opus 5.5
Reviewer to evaluate first	1 bounded Opus 5.5 reviewer on frozen repository changes
Effort	Explicit medium, then a measured comparison with high
Escalation to test	Fable 5.1 on the cases Opus 5.5 misses at higher effort
Extra reviewers	Keep them when they catch verified defects the baseline misses
Harness migration	Map thinking blocks, refusals, forced tool calls, and fallbacks at the adapter
Success measure	Verified defects, false alarms, reviewer time, and total attempt cost

The review loop described in parts 1 to 7 needs a concrete model to start from. For the next evaluation, that model is Opus 5.5, tested as 1 bounded reviewer before any second reviewer is added. This article explains what the launch evidence does and does not support, which harness changes the model requires, and what the evaluation should record.

Anthropic released Opus 5.5 on September 22, 2026. Its launch page reports 66.4% on Terminal-Bench 4.0 at xhigh effort, against 55.8% for Fable 5.1 at max. It reports a 40% cost reduction versus Opus 5 at default settings on its typical workloads. The model overview, checked on October 14, 2026, recommends Fable 5.1 when evaluations of Opus 5.5 at higher effort still fall short.

The launch page also carries a Deloitte testimonial claiming that low-effort Opus 5.5 caught 72% of known bugs in its reviews, versus 56% for high-effort Opus 5. The corpus and adjudication record are not supplied there, so this is a customer statement carried by the vendor, not an independently reproducible review comparison. These sources justify candidates to test; they do not settle which configuration fits our repository. Fable 5.1 remains an escalation candidate subject to access and evaluation.

Start with 1 reviewer and a useful question

Model references checked on October 15, 2026

The current Claude model overview lists claude-opus-5-5, claude-fable-5-1, claude-sonnet-5-5 and claude-haiku-5-5. It gives Opus 5.5 a default effort of medium and Fable 5.1 a default of high. Sonnet and Haiku are available family members, but this article does not evaluate them as reviewers. A model listed by a provider is not proof of access through our account or deployment route.

The Opus 4.8 and Sonnet 5 names in the September pilot below remain historical. Updating a candidate list does not change the models that produced an earlier result.

A review system can accumulate agents without accumulating evidence. One agent searches for bugs, another repeats the search, a third summarizes their overlap, and a developer still decides which comments matter. A more capable model is a reason to test whether part of that machinery can go.

The first question is whether 1 Opus 5.5 reviewer finds the required defects, explains them well enough to act on, and completes the declared scope within budget. The second is what a separate reviewer adds on the cases the baseline misses. A security specialist, a deterministic check, and another general reviewer are different interventions, so give each one a defined property to inspect.

Part 7 shows why the question matters: on 1 repository with 5 AI reviewers, 4.5% of inline comments received a visible human response. Before comparing candidates, establish the reference defects before any output is seen, as part 7 describes, and include changes with no known defect as well: a reviewer that always finds something to say creates work even when its positive findings look impressive.

Calibrate effort before adding more agents

Anthropic’s Opus 5.5 prompting guidance recommends starting at medium, the model’s default, setting it explicitly, and testing other levels against your own evaluations. It also warns that effort names do not correspond to the same amount of thinking across models: in Anthropic’s testing, Opus 5.5 at medium matches or exceeds Opus 5 at high on coding and knowledge-work evaluations. Carrying an old high or max setting forward can change the cost and duration of every review.

Use the same qualified changes for these conditions:

Condition	Purpose	Evidence needed
Opus 5.5, medium	Establish the operating baseline	Completed scope, verified findings, omissions, cost, and latency
Opus 5.5, high	Test whether extra effort pays for itself	Additional verified defects and their review cost
Fable 5.1 on the residual misses	Test the escalation Anthropic recommends	Defects recovered and the added cost per recovered defect
Opus 5.5 plus 1 independent reviewer	Test the value of another review path	Unique defects rescued after deduplication and adjudication

Keep the prompt, permitted tools, initial application state, and corpus fixed across the effort comparison. For the last condition, count both reviewers and the aggregation work. A system that spends twice as much has to show what the extra spend buys.

Choose the acceptance threshold before running the reserved set. It can differ between a documentation edit and a payment calculation, but it cannot be rewritten after a favored configuration loses. Report failed and partial attempts beside successful ones. A fast review that never reaches the affected file has not earned a latency advantage.

Make the adapter compatible with Opus 5.5

The migration guide, checked on October 14, 2026, lists changes for a custom Messages API client:

• adaptive thinking is always on, and max_tokens covers thinking plus response text
• forced tool_choice types any and tool return a 400 error; use auto and validate tool inputs
• thinking text is omitted by default; request thinking.display: "summarized" when visible progress is needed
• return the assistant message, including thinking blocks, complete and unmodified when supplying tool results
• a refusal can return HTTP 200 with stop_reason: "refusal" and a category in stop_details
• configured server-side fallback can change the model answering a declined request

Strict tool use and structured-output support differ by deployment platform; the documented Bedrock path uses auto without strict. Parse content blocks by type rather than assuming content[0] contains text. These are documented API contracts, not an adapter I have exercised in this article.

Claude Code handles its own integration. A custom client needs the mappings below, which translate these behaviors into the execution states and stop causes of part 3:

Observation	Loop action
Text-only end of turn, required checks outstanding	Inspect the checklist; continue only within the remaining budget
Background command or reviewer still running	Keep the run active and collect its result
HTTP 200 with stop_reason: "refusal"	FAILED with REFUSED; never emit CLEAR
No visible progress text	Read the supported block types; infer neither inactivity nor completion
Valid findings object, incomplete scope	FAILED with PARTIAL_SCOPE; keep the findings as evidence
A configured fallback produced the result	Record the model that answered and report the run separately from the Opus 5.5 condition

Give the adapter fixtures for a valid empty finding list, malformed findings, a refusal, an exhausted budget, and an interrupted tool sequence. This article proposes those acceptance checks; it does not claim that an adapter has passed them. A supported fallback is still subject to the workflow’s own authorization policy.

A finished turn still needs a task outcome

A reviewer can describe what it has done and announce its next step without completing that step. The orchestrator needs a retained checklist of required work, not a guess based on the tone of the final paragraph.

When a turn ends, inspect outstanding obligations and background work. If the checks are complete, validate the terminal report. If work remains and the run may continue, name the missing work in a bounded continuation. Refusals, missing permissions, and exhausted budgets stay explicit stop causes; a continuation policy must not turn them into an invitation to route around the boundary.

For the initial design, I would allow at most 2 automatic continuations within a review pass and require new evidence of progress between them. That is a proposed operating limit, not a measured optimum. It stops the run with a defensible outcome, or with enough information for someone to resolve the blocker.

What the previous pilot still teaches

An earlier pilot, which I ran in September on Opus 4.8 and Sonnet 5 and have not published, made 48 attempts on 12 pull requests. It exposed 5 incorrect diff boundaries. In 1 case, a review received 1,996 changed files where the corrected base produced 16. The preparation also omitted some relevant source files. None of its results transfer to Opus 5.5.

The lesson worth keeping is a preparation gate. Record the selected base and head, the expected changed files, the actual diff files, the available full sources, and any missing context, and stop on an unexplained mismatch. A stronger reviewer might still find a real bug in an oversized packet, but that finding needs a separate disposition and says nothing about coverage of the intended change.

Measure what reaches an accepted change

Each review record identifies claude-opus-5-5, the deployment provider, the client version, the effort, and the model that actually answered each attempt. A fallback changes the treatment, so report mixed-model runs separately from the Opus 5.5 condition.

For each condition, retain verified defects, false alarms, omissions against the hidden reference, complete and partial outcomes, elapsed time, all-attempt cost, and the human time spent resolving findings. A second reviewer that catches 1 serious omission can be worth its cost; one that produces 20 duplicate comments makes the workflow worse.

Inspect AI, maintained by the UK AI Security Institute, offers a documented evaluation structure with separate datasets, solvers and scorers, plus logs for examining individual runs. It is one implementation to inspect if the proposed reviewer comparison grows. Choosing a model-based scorer still requires checking its judgments against an independent reference. I have not integrated Inspect into the Luna/Astra preparation, and its available features do not validate our corpus or supply Opus 5.5 results.

The Claude API price sheet, rechecked October 15, 2026, lists Opus 5.5 at $4 per million uncached input tokens, $20 output, $0.20 cache reads, $5 five-minute cache creation and $8 one-hour cache creation. Apply the rates for the actual deployment route and token classes. Subscription quota consumption stays separate from an API-equivalent estimate; part 5 of The Real Cost of AI gives the current cross-provider snapshot.

The deployment decision comes from these retained results. The role of Opus 5.5 in the fleet, its effort, and its allowed actions follow what the qualified corpus shows, and part 1 still applies: a better reviewer only helps if the stage it serves is the one where work waits.

Separate preparation from model results

Preparation status, October 14, 2026.

A separate coding pilot is prepared for Codex: 3 frozen tasks, each assigned once to 4 configurations, making 12 planned attempts. The model identifiers come from the installed client’s catalog:

Model	Requested effort	Planned attempts
gpt-6-luna	medium	3
gpt-6-luna	xhigh	3
gpt-6-astra	medium	3
gpt-6-astra	xhigh	3

The preparation has passed 31 local tests and 6 operating-system isolation checks. Four sessions exercised the installed Codex client against a local server returning fixed responses. They checked request fields and synthetic token counters; interrupting a held response disconnected its local TCP stream, and a subsequent continuation was refused. Twelve confined evaluations used operator-written control submissions, including checks that rejected 3 deliberately faulty implementations. These are preparation checks, with no model-generated output.

No benchmark model has been called, and no execution budget is approved. Actual model access, provider-applied effort, subscription consumption, billing, final usage after interruption, a hard consumption limit during a response and a memory ceiling remain unverified. The Codex app-server protocol describes the client interface; passing a local simulation does not qualify a live provider.

This coding pilot does not test reviewer recall, reproduce Jain’s service-building experiment or evaluate the Opus 5.5 adapter above. One attempt per task and configuration would be a pilot, not enough evidence for a model ranking. Budget approval and technical admission still precede any real run.

---

YSNK

(You should now know)

• Opus 5.5 is the baseline to test in this review workflow, with Fable 5.1 as the escalation condition
• Launch benchmarks are vendor evaluations of coding work, not code-review recall
• Compare effort levels before paying for a larger fleet
• The API adapter must map thinking blocks, refusals, forced tool calls, and fallbacks explicitly
• A current decision requires current runs on a qualified corpus