Coding faster moves the cost to verification

TL;DR

I would measure where a change waits and returns for correction before adding another authoring agent. Faster code generation helps delivery only when verification and release can handle the result at an acceptable quality and total cost.

Decision	What to inspect
Treat faster coding as a local speedup	Track specification, review, CI, and deployment, then measure operational stability and the intended product outcome separately
Find the queue, not the busiest person	Split elapsed time by stage and separate active work from waiting
Keep DORA as the delivery outcome layer	Add review, CI, and rework signals to explain why service-level throughput or instability changed
Treat the workflow example as a hypothesis	A constructed scenario explains what to measure; it supplies no observed volume, saving, or before/after result
Make the Build decision from evidence	Instrument 1 service before adding another authoring agent or another review layer

Faster code generation is a local speedup. Shipping is a property of the whole delivery system.

AI can shorten the authoring step without changing the rate at which a team specifies, verifies, reviews, integrates, and releases changes. If generated code reaches those stages faster than they can process it, the saved authoring time reappears as a queue: the team produces more candidate changes while lead time stays flat, CI runs multiply, or reviewers absorb the difference.

DORA published direct evidence for this pattern on March 10, 2026. Its researchers ran a thematic analysis of 1,110 open-ended responses from Google software engineers, collected in Q3 2025, about the effect of AI tools on their workflow over the preceding 3 months. Respondents often described time saved during creation being spent again on verification and prompting. DORA notes that earlier survey questions may have primed respondents to discuss code generation. The source shows that engineers at a single company reported this cost shift. It does not show that every AI-assisted team has the same bottleneck.

Code can still be the constraint. A developer learning an unfamiliar system, implementing a difficult algorithm, or working without useful automation may spend most of the cycle producing the change. Once authoring gets faster, the next job is to find the stage that now controls delivery. Lean and AI follows that question beyond software delivery to the user or service outcome.

How to read the evidence in this series. Each claim carries 1 of these labels, in words or in a status column. Documented means a primary source states it. Scoped evidence means a study or survey supports it within its own protocol. Author’s statement means a person describes their own design or experience. Vendor statement means a company describes its own product. Constructed example means a scenario built to explain what to measure. Proposed protocol means a method this series recommends and has not yet executed. Not measured means no record exists yet.

A pull request covers only part of the delivery flow

A change crosses several boundaries before its operational and product effects can be judged:

1. The team agrees on intent and acceptance criteria
2. The author or agent produces a candidate change
3. Local checks test syntax, types, behavior, and policy
4. The author declares the change ready for review
5. A reviewer understands the intent and evaluates the diff
6. CI runs the checks that belong in a reproducible environment
7. An authorized person or policy permits merge and deployment
8. Production evidence reveals whether the change caused corrective work
9. The team observes the predefined user, service, or business outcome during its stated window

Each boundary can contain active work and waiting. A review may be brief after a much longer queue, and a CI job may spend more time waiting for a runner than executing. Collapsing both intervals into “PR time” tells you that delivery is slow and hides what to change.

The authoring step has the same ambiguity. A coding agent can produce a diff quickly, then spend several cycles repairing tests, resolving reviewer comments, or reconciling a vague requirement. Counting only the first diff rewards early output and omits the work required to make it acceptable.

Lines generated, commits created, prompts sent, and pull requests opened describe candidate output. A deployment records a delivered change. Change fail rate and deployment rework rate describe operational stability. A predefined user, service, or business measure describes the intended product outcome. Movement at 1 layer does not establish movement at the next.

Each passage should leave an artifact that the next actor can inspect: acceptance criteria, a plan with unresolved questions, code and tests, a review record, and release evidence. The arrows also run backward. Critique revises the plan, a confirmed review finding changes code or tests, and an adjudicated lesson proposes an update to the instructions or examples used by the next task, which is validated before distribution.

DORA gives the outcome layer, not the full diagnosis

The current DORA software delivery metrics, last updated by DORA on January 5, 2026, use 5 measures for 1 application or service:

• Change lead time, deployment frequency, and failed deployment recovery time describe throughput
• Change fail rate and deployment rework rate describe instability

These measures stay useful when AI assists the work. Change lead time runs from commit to production, so it captures review, CI, and deployment delay after the commit. Deployment rework rate counts unplanned deployments caused by production incidents. Together, the 5 measures show whether the service is moving changes safely and quickly.

They do not show which pre-deployment stage absorbed the time. Change lead time excludes specification and any work performed before the first commit. A stable deployment frequency can hide a growing review queue, and a short merge time can hide a shallow review. DORA’s own implementation guidance recommends mapping the delivery process and adding leading indicators, such as code-review duration or test quality, when diagnosing the main constraint.

Use DORA as the outcome layer, then add workflow signals that explain movement in those outcomes, without swapping one proxy for another. A review-time target can be gamed by approving faster, just as a lines-of-code target can be gamed by producing larger diffs.

Claim	Status	Boundary
DORA currently defines 5 delivery metrics and applies them at service level	Documented	Official guide updated January 5, 2026
Engineers reported creation time moving into verification and prompting	Scoped evidence	1,110 open-ended responses from Google engineers, qualitative
Faster authoring can increase the verification queue	Constructed example	The scenario below is not an observed team result
Moving selected review before remote CI reduces total delivery cost	Not measured	Requires linked local, remote, and rework records under a comparable acceptance rule

Test the hypothesis on a single delivery flow

Consider a constructed example: an agent prepares changes faster, while reviewer availability and CI capacity stay fixed. The team corrects some defects only after a push, and each revision triggers another remote test run.

The example suggests 2 hypotheses. Faster authoring may raise the arrival rate at verification, and moving selected checks before the push may reduce avoidable remote runs. Neither establishes a lower total cost. Local checks consume resources too, and fewer remote runs are no gain if defects escape or if authoring and human review become more expensive.

Evidence required for a measured case: define the observation period, unique changes and contributors, size distribution, stage-by-stage lead time, local and remote execution costs, and the rule for attributing post-merge rework. Compare the same service under the same acceptance criteria and disclose concurrent workflow changes.

The scenario helps choose which records to collect. Only those records can support a claim about a team’s bottleneck or savings.

Use review failures to change upstream work

In IFTTD episode 362, Yacine Hmito describes faster implementation moving pressure onto quality control. His Lean framing treats review as a way to learn why defective work appears and to change how it is produced, instead of letting code accumulate while it waits for inspection. Dex Horthy, around 6:20, says that agentic code review and regression testing make that stage faster while it probably remains the bottleneck. Both are practitioner accounts, not throughput measurements.

Toyota’s description of its production system makes the analogy more precise. Just-in-Time makes only what the next process needs, when it needs it. Jidoka detects an abnormality, stops the affected work and changes the process to prevent recurrence. Applied here, the team limits new authoring to what its reviewers and CI can verify, pauses new intake when that capacity is exhausted, and turns confirmed recurring defects into tested controls. An alert without a stop rule only makes the queue visible. This is a proposed software adaptation, not a measured Toyota-equivalent result.

A longitudinal study of one enterprise AI-coding mandate adds a useful observation without supplying a universal capacity ratio. Across 802 developers and 196,212 pull requests, per-capita merged-PR throughput reached 2.09 times the pre-mandate level in April 2026, while the load per reviewer roughly doubled. The authors report stable merge and revert rates, but do not claim exact causal attribution to AI adoption or measure the value delivered by those PRs.

Bruno Soulez’s IFTTD season conclusion (archived September 13, 2026) proposes starting with validation before expanding generation, and presents that ordering as his own reading. The operational version is narrower: pause new authoring when the shared verification queue exceeds its agreed capacity, keep existing reviews and corrections moving, and resume under a stated condition. A budget on each agent run does not bound the total queue.

Part 5, Turn recurring review comments into tested controls, covers the upstream response: accepted corrections that keep returning become candidates for tests, examples, or rules, each with its own validation.

Measure the queue and the correction cycles

A useful first dashboard is small enough to maintain and detailed enough to locate waiting. Define every timestamp before collecting it. “Ready for review” in particular should be an explicit event, not the time the branch or pull request was created.

Signal	Definition	What it can reveal	What it cannot prove
Specification time	Acceptance criteria agreed minus work requested	Ambiguity or decision delay before implementation	Specification quality
Authoring time	Ready for review minus implementation start	Active and elapsed effort before review	Value delivered or correctness
Review wait	First substantive review minus ready for review	Queue depth and reviewer availability	Review quality
Review cycles	Count of change requests followed by a new revision	Reconciliation load and unstable requirements	Which party caused the rework
CI queue and execution	Runner start minus request, plus runner end minus start	Capacity wait and actual compute duration	Whether the checks cover the right behavior
CI reruns per change	Completed runs linked to the same pull request and revision history	Repeated remote feedback cycles	Which reruns were avoidable without classifying them
CI cost per change	Provider charges or billed runner time allocated to the pull request and commit SHA, retaining workflow and runner class	Spend attached to each delivered candidate	Avoidable spend until reruns are classified by cause
Test-selection data freshness	Result availability to the test-selection service minus CI run completion	Whether test-selection decisions use stale history	That the selected tests cover the right behavior
Merge and deployment wait	Merge or deployment minus all required evidence ready	Approval, release-window, or environment delay	Product value
Deployment rework rate	DORA’s ratio of unplanned deployments caused by production incidents	Corrective delivery work after production failure	Root cause without incident linkage
Intended product outcome	1 user, service, or business measure defined before the change for a named population and observation window	Whether the intended outcome moved during that window	Causality without an appropriate comparison design

For time measures, inspect the median and a tail percentile rather than the mean alone. A few stuck changes can dominate an average, while the median alone can hide the queue that hurts the riskiest work. Segment the view by change size and risk class. If AI attribution is available, publish its definition and coverage before comparing assisted and unassisted changes, because a label applied inconsistently creates a false control group.

Review cycles need classification, not blame. Separate at least specification corrections, implementation defects, test or policy failures, and reviewer disagreement. The categories show whether to improve the input, add a deterministic check, clarify ownership, or change the review contract.

CI cost needs the same discipline. Link each run to a pull request, commit SHA, workflow, runner class, and terminal status. A green job proves that its declared checks passed on that SHA. It does not prove that the review covered the full change, that the job was required for merge, or that the product behavior was correct.

When CI is constrained, test selection can reduce repeated work. Anthropic’s September 14, 2026 account describes a listener recording results and a selector using that history. Doubling cores lasted 70 days, sharding 29, and restarts less than a day. A 20-minute listener lag could leave tens of thousands of updates unapplied, including new or fixed tests. That motivates recording selection-data freshness separately from whether CI ran.

Its chart shows the queued-result backlog flattening after the cutover. The author describes the new architecture as more expensive to run and reports 3 weeks of work by 1 engineer, without a numerical delivery-cost comparison. The chart does not measure faster or cheaper delivery.

Admit work only when it can be verified

Johan Martinsson’s account of reviewing AI-written pull requests connects a growing review queue with a loss of creative work. It does not establish a general productivity loss, but it raises a capacity question: who can verify the next result, with which evidence, and in what time?

Before increasing parallel authoring, name that verifier and use their declared availability. Admit, reduce, or defer the task. A deferral needs an owner, a reason, and a condition for resuming, and work deferred repeatedly needs an explicit decision.

Include failed attempts in the cost of accepted work

Are Harnesses Worth the Cost?, Naresh Jain’s October 13, 2026 experiment, makes that accounting problem concrete. On one Java service, ten runs per condition, low reasoning effort and identical external acceptance tests, a behavioral harness raised Luna’s acceptance from 3/10 to 10/10. Sol accepted 10/10 with or without it while consuming more resources with it. The reported generation costs exclude setup, maintenance and repairs. This is evidence for those conditions, not a measurement of our review workflow.

The Scaffold Effect runs 2 models, Qwen 3.6 Plus and MiniMax M2.5, through 3 harnesses on 50 Terminal-Bench Pro tasks and finds up to a 40× difference in tokens per solved task depending on the harness. That is an efficiency result under its protocol, not a saving in any team’s bill or review time.

Define the accounting window before calculating cost per accepted change. Include failed and abandoned attempts in the numerator, and count each accepted change once in the denominator. Keep inference charges, CI and infrastructure, human review, and rework separate until a declared conversion rule makes them comparable, and report the acceptance rule and sample size beside the result. If no change is accepted, report that outcome and the resources spent rather than hiding the run behind a cheap average per call.

Compare a service against its own baseline

Measurement can change who and what enters the comparison. In its February 24, 2026 experiment update, METR reports that developers increasingly declined tasks they would have to attempt without AI, while parallel agent work complicated time reporting. It treats the newer data as weak evidence for the size of a speedup because of these selection effects. The earlier 19% slowdown remains a result about early-2025 tools and its studied population. My comparison of the Anthropic and METR results keeps their different outcomes separate. For a review-loop experiment, record excluded tasks and overlapping human work before interpreting the observed duration.

Start with 1 application or service, following DORA’s scope. Capture the current flow before changing the workflow, then introduce 1 bounded intervention, such as a deterministic local check before the first push or a limit on reviewable batch size.

Compare distributions, not a single headline number. A useful result states the observation windows, the number and size of changes, missing events, changes in team composition or policy, and the definition of every stage. If review wait falls while authoring time rises slightly, the total may still improve. If CI spend falls while deployment rework rises, the team removed cost from the wrong place.

Count the friction the harness adds in the same comparison:

Added cost	What to retain
False blocks	Valid changes stopped, the control responsible, and time to resolve them
Redundant approvals	Repeated confirmation of an unchanged action with no new evidence or authority boundary
Context overhead	Loaded instructions, irrelevant material, and any observable effect on task completion
Maintenance	Time spent updating, diagnosing, evaluating, and retiring the controls

Use the same acceptance criteria for the baseline and the candidate workflow. A protection invoked rarely may still enforce a necessary invariant, so keep its permitted and forbidden examples and its owner; a low invocation count alone is not grounds for removal.

Correlation is enough to locate a candidate constraint, not to assign cause. A new reviewer, a release freeze, fewer contributors, smaller pull requests, or a CI configuration change can all move the same metric. Record those changes beside the chart instead of writing a causal story after the fact.

This comparison also gives the stop rule for adding generation. If the authoring queue is empty while review wait, correction cycles, or CI retries rise, another authoring agent only raises the arrival rate at the constrained stage, so improve or bound verification first. If authoring still dominates end-to-end time and downstream signals stay stable, more authoring capacity may help.

Decide which judgment to delegate

Once the queue is visible, the next question is who judges what. Tangi Vass, who created Liza, gives the project’s answer in its README: agent deliberation is abundant while human attention is scarce, so Liza’s contract “relocates that work into agent reasoning and review, so human attention can go to direction and domain decisions.” In a LinkedIn post, he describes a first pass as 1 autonomous delivery run that includes its internal correction cycles. Both describe his design; neither compares escaped defects or review cost.

Any allocation needs a policy for which changes can advance and evidence about what that policy accepts. A small diff can change authorization or interact with another small PR, so classify the combined change and its recovery path rather than the number of lines. Keep human attention on domain decisions and on consequences that the current tests and reviewers have not shown they can cover. The guide’s section on allocating judgment explicitly develops that boundary.

Product acceptance is a separate question from technical review. In Practical Loop Engineering, Addy Osmani recounts asking agents to research competitors and prepare candidate changes, then judging that the added complexity for users outweighed the gain. A change that can be implemented still needs someone to decide that it belongs in the product, before completed agent tasks are counted as useful output.

Software factories inherit the same constraint

A software factory can coordinate many authoring runs, but its throughput still depends on what accepts or rejects their output. More agents increase the number of candidate changes. They do not add reviewer attention, executable specifications, runner capacity, or release authority.

The same account reports 10x as many tests and 25x as many CI jobs over 6 months, with limited staff growth. Not every test runs on every PR, so the figures do not multiply. Overnight agent activity also raised the baseline load. The account supplies no capacity ratio another team can apply.

Verification therefore belongs in the system design, at 3 horizons that the rest of this series uses. The inner loop catches cheap failures while an agent works. The outer loop evaluates the change at the task or pull-request boundary. The meta loop turns recurring failures into tests, rules, or other durable controls. These horizons help place feedback; they do not guarantee that the feedback is correct.

The Build-stage decision is to instrument the flow before expanding production: find the stage where work waits, measure what returns for correction, and define the evidence that allows a change to advance. Part 2, When an independent reviewer earns its place, examines 1 verification boundary in that system without treating a second model as proof of correctness.

---

YSNK

(You should now know)

• Faster authoring can move delivery cost into specification, review, CI, and rework
• DORA’s 5 metrics show service-level throughput and instability but need workflow signals for diagnosis
• Candidate output, delivered change, operational stability, and product outcome require separate measures
• Constructed scenarios and installed controls do not establish a measured bottleneck or saving
• Instrument 1 service and its correction cycles before adding more authoring capacity