AI coding productivity: why one gain is not your forecast

TL;DR

AI coding can pay off, but 20-30% is not a default business-case forecast. That range comes from a DX practitioner’s account; Yegor Denisov-Blanch presented a 15-20% net estimate after rework in a research talk. Two 2026 field studies measured more merged pull requests under particular rollout conditions, without measuring the value of those changes. A CTO’s budget needs a local baseline, total acceptance cost and an intended product outcome.

Claim	Number	Source
Net productivity estimate after rework	15-20%	Yegor Denisov-Blanch’s 2025 research talk, which describes a research program covering over 100,000 engineers and a separate pilot of about 120 developers; no complete method for the headline estimate is supplied here
Apparent output gain before rework	30-40%	Same talk; not a product-value measure
Practitioner’s velocity estimate	20-30%; Intercom reported 41% more AI-driven developer time savings	Justin Reock, DX, August 2025; the Intercom measure has a different denominator
Merged-PR change after CLI-agent adoption	About 24% more among adopters	Murphy-Hill et al., Microsoft rollout; merged PRs are an output proxy, not delivered value
Merged-PR throughput and review load in one company	2.09 times the pre-mandate PR baseline; roughly twice the load per reviewer	He et al., enterprise case study; non-random adoption and no exact causal attribution
AI-generated PR merge rate vs. human	60% vs. 80%	Nick Arcolano, Head of AI & Research at Jellyfish, 250,000 developers tracked, June 2026
”2x requests might mean 2x bugs”	Independent warning, same conclusion	Thomas Bentkowski, Product Manager at Doctolib, September 2025
Floor case, insufficient review	Rework reported, no net figure given	Max Sumrall, Software Engineer at Picnic, Devoxx, April 2026
Ceiling case 1: fully autonomous PRs	1,300/week as reported at the time, human review still mandatory	Steve Kaliski, Stripe, corroborated on stripe.dev, Stripe Sessions 2026, and the How I AI podcast
Ceiling case 2: one team’s throughput jump	3.5 to 70 PRs/engineer/week, GPT-5.2 to GPT-5.5 (GPT-6 family not covered)	Ryan Lopopolo, Member of Technical Staff at OpenAI, June 2026 account, “on my team”

In a 2025 research talk, Yegor Denisov-Blanch described longitudinal Git data covering more than 100,000 engineers across over 600 companies. He used a separate pilot involving about 120 developers to show why added output and recent-code rework must be counted separately. His presentation gave an apparent output gain of 30-40% and an aggregate estimate of 15-20% after rework.

The talk makes the measurement distinction visible, but it does not publish a complete study protocol for reconstructing the headline estimate. Its figures should be attributed to the speaker’s presentation, not turned into a controlled, transferable return for every team. The research concerns activity visible in Git; it cannot, by itself, establish customer value.

DX describes another measure, not a second estimate of the same outcome

In an August 2025 conversation, Justin Reock, Deputy CTO at DX, put a number on what he sees across customers: “more realistically, looking at numbers like 20, 25, 30% increases in velocity, I think is a good number, and it’s certainly making an impact.”

He drew a hard line against anything bigger. “It’s not the 100x improvements,” he said. “That’s a lot of information that’s very good for creating YouTube subscribers and upvotes, but not necessarily increased productivity for organizations.”

One customer beat that range by a wide margin. DX’s public data shows Intercom achieved a 41% increase in AI-driven developer time savings, which Reock characterized as an outlier rather than a typical result.

These numbers do not independently confirm the Stanford estimate. Reock discusses velocity, while Intercom’s 41% concerns AI-driven developer time savings. Neither supplies a general distribution of net business outcomes. Before using 20-30% in a budget, a team needs to define which measure would improve, how rework is counted and which product result would make the spend worthwhile.

Two field studies count merged pull requests

Murphy-Hill and colleagues studied Microsoft’s early-2026 rollout of Claude Code and GitHub Copilot CLI across tens of thousands of engineers. Their analysis associates adoption with roughly 24% more merged pull requests than adopters would otherwise have merged over the four-month observation window. The authors identify merged PRs as an output proxy and explicitly distinguish them from the value those PRs deliver.

A second longitudinal study follows one AI-forward company through a mandate to double merged PRs per engineer. In a panel of 802 developers and 196,212 PRs, per-capita throughput reached 2.09 times its pre-mandate level in April 2026. The load per reviewer roughly doubled and automated review overtook human review; merge and revert rates held steady. Adoption was not randomly assigned, so the authors do not attribute that entire change to AI. Stable merge and revert rates also leave product value and longer-term maintenance unmeasured.

The two studies establish that more PRs can be merged in these settings. They do not convert PR throughput into a net productivity percentage comparable with Denisov-Blanch’s estimate, and neither can determine a team’s budget from its headline number. The next measurement belongs at the service and product level: review effort, corrective work, operational stability and the intended user outcome.

Where the volume goes wrong

Nick Arcolano, Head of AI & Research at Jellyfish, sits on a dataset that tracks 250,000 developers across the industry. In a June 2026 conversation, he laid out a failure mode that helps explain why volume overstates output: AI-generated pull requests get merged at a lower rate than human-written ones.

“The average for Q1 for humans was about 80% of pull requests that were open ultimately got merged,” he said. “What we see with agentic PRs is 60/40 instead of 80/20. So you’re talking about double the amount of PRs that are kind of dying on the vine.”

Twice the volume of code doesn’t translate into twice the shipped output when a much larger share of that volume never survives review.

A different team reached the same conclusion from the opposite direction, nine months earlier and with no connection to Jellyfish’s dataset. Thomas Bentkowski, Product Manager at Doctolib, described his own team’s internal metric in September 2025: “if we are producing twice as many requests compared to three months ago, it means that we double our productivity, right? Well, no, because we might as well have doubled the number of bugs as well. So doubling the number of requests alone doesn’t mean anything.”

A vendor dataset and a product manager’s account give the same warning from different positions. Raw request volume is a vanity metric until someone checks what happens to it downstream.

Bentkowski’s own team gets the deepest check this series runs on any single company: Part 7 follows Doctolib’s 30-to-600 rollout through two corroborating talks (four Doctolib talks in all), then separates it from three other Doctolib initiatives documented elsewhere.

The floor, when nobody checks

Volume without enough review has a reported floor. At Devoxx, Max Sumrall, a software engineer at Picnic, described what happened when his team let AI-generated code ship without enough scrutiny in April 2026: “the code is shipped and then it needs to be touched again and again and again to actually make it work.”

Sumrall describes the same kind of rework cost the Stanford study measures, without reporting a net figure.

Picnic’s account shows a possible rework mechanism, not where the team sits relative to the 15-20% estimate. The speaker reports repeated corrections but no comparable net productivity figure. The account does not isolate model quality from review practice.

The two numbers that don’t belong on the same chart as the rest

Two figures from this same body of reporting run far past 15-20%, and both are cited as if they represented the norm rather than the extreme.

Stripe’s internal coding agent, called Minions, was reported at the time to generate roughly 1,300 pull requests a week without a human writing the code, according to Steve Kaliski, a Stripe engineer who described the system on three separate occasions: Stripe’s own engineering blog, a keynote at Stripe Sessions 2026, and an episode of the How I AI podcast.

All three converge on the same number and the same caveat: humans stop writing the code, but human review before merge stays mandatory.

What Stripe claims is narrower than autonomy: a coding phase with no human keystrokes, sitting in front of a review phase that hasn’t gone anywhere.

The second figure comes from one specific OpenAI team. Ryan Lopopolo, a Member of Technical Staff at OpenAI who is associated with the term “harness engineering,” described his own team’s trajectory in a June 2026 conversation: “at the beginning of sort of the 5.2 era, we were looking at maybe three and a half PRs per engineer per week on my team. Now with 5.5, we’re looking at 70.”

That’s a jump attached explicitly to one team building one internal product, tracked across two model generations.

The account names GPT-5.2 and GPT-5.5 and covers no later model, including the GPT-6 family OpenAI now lists. It also does not isolate the effect of the model upgrade from changes to the harness and workflow.

It says something real about what harness engineering can produce inside a specific, deliberately instrumented setup. It says nothing about what an average OpenAI engineer, or any other company’s engineer, should expect to see on their own dashboard next quarter.

Both numbers are true for their scope, and both lose it when repeated on conference stages, until “one OpenAI team going from 3.5 to 70 PRs a week” quietly becomes “OpenAI is 20x more productive now.”

A third case arrived in September 2026 with the same shape and one difference worth keeping. In an engineering account of its own CI load, Anthropic reports that its engineers “on average ship 8x as much code per quarter as they did from 2021-2025” and that “Claude authors 80% of that code and it also plays a large role in reviewing and approving PRs as well”.

No counting methodology is published for the 8x, and “code per quarter” carries no stated unit, so the multiple cannot be reconstructed or lined up against Stanford’s rework-adjusted 15-20%.

The difference is that the thing being measured is code volume, and Anthropic says so rather than letting the reader infer productivity.

Addy Osmani, writing about the same numbers, keeps the caveat that usually falls off on the third repetition: “Lines of code isn’t (of course) a direct measure of value.”

An honest volume measure is still a volume measure. It belongs next to Stripe’s 1,300 weekly pull requests (as reported at the time), not next to a merge rate or a net gain.

The gap that sets up the next question

Put the registers side by side. Denisov-Blanch presented a 15-20% aggregate estimate after rework. DX described a 20-30% velocity range. Microsoft measured about 24% more merged PRs among CLI-agent adopters, while one company in the enterprise mandate study reached 2.09 times its earlier PR throughput with about twice the reviewer load. These measures have different populations and denominators; none is an observed return on a new team’s budget.

The case-specific extremes are Stripe’s 1,300 autonomous pull requests per week (as reported at the time) and one instrumented OpenAI team’s 20x jump from 3.5 to 70 pull requests per engineer per week.

Neither number is fake. Both are being spent against, right now, by companies deciding how much budget an AI coding rollout deserves.

Part 1 of this series tracked what an agent costs to run once volume shows up. Part 2 showed why almost nobody can put a verified number on the return side of that spending.

For a CTO business case, start with the team’s own baseline and a bounded pilot. Count the complete cost of accepted work, including abandoned attempts, review and rework, then observe a predefined service or user outcome. The external figures help choose what to measure and where to look for a constraint; they cannot supply the result of that local comparison.

AI velocity is bidirectional covers the twin problem on the delivery side, shipping faster and accumulating debt faster in the same motion.

The next part turns to FinOps: who owns the AI coding budget, and how routing decisions get tested against it.

---

YSNK

(You should now know)

• The Stanford research talk presents a 15-20% aggregate estimate after rework, but does not provide a complete protocol here for transferring that figure to another team
• DX’s 20-30% velocity estimate and Intercom’s 41% increase in AI-driven time savings use different measures; neither confirms a default business-case return
• Two 2026 field studies report more merged PRs in specific rollouts, about 24% among Microsoft adopters and 2.09 times the pre-mandate rate in one company whose reviewer load roughly doubled; PRs are not product value
• AI-generated pull requests merge at a lower rate than human-written ones, 60% versus 80% per Jellyfish’s 250,000-developer dataset, which is why raw request volume overstates real output
• Stripe’s 1,300 autonomous PRs a week (as reported at the time) and one OpenAI team’s jump from 3.5 to 70 PRs per engineer per week are both real, and both are case-specific extremes, not the industry norm
• A low floor exists too: Picnic’s Max Sumrall reported AI-generated code that shipped without enough review and needed repeated rework