Is AI-written code buggier than human code? We blamed 112,000 commits to find out

No. Across 112,382 commits in 28 public repositories, AI-agent commits were not more likely to introduce a bug than human commits in the same repository, the point estimates were protective, and agent-written lines survived longer. We measured this with SZZ, the standard method that traces each bug fix back to the commit that planted the bug, and controlled for the size of every change. The effect is modest and lives in the human-driven tier, where a person still reviews the diff.

Everyone has a prior on this question and few have data. The measurements that exist are mostly surveys of how developers feel about AI code, not what the code does once merged. So we went back through the git history of real projects, covering the first year of coding agents merging real pull requests, and asked one thing: when an agent wrote the commit, was it more likely to be the one that introduced a bug?

What counts as agent code

"AI wrote it" is not one thing. A senior engineer steering Claude Code through a refactor and an unattended bot opening scheduled PRs are both AI code, and pooling them hides the signal. So we built a per-commit provenance detector that reads eight signals: bot account identities, service email addresses, commit-message footers, co-author trailers, and merged-PR evidence such as agent branch prefixes and PR-body markers. (For how provenance detection compares with trailers and session recorders, see AI code provenance: how to tell which code agents wrote.)

We blind-validated the detector on 124 commits with six independent reviewers. It reached 96.2% precision, with six of the eight channels perfect. The one real failure mode, a human pushing a follow-up commit inside an agent's PR, is downgraded in confidence so it can be filtered.

Agent commits were split into three tiers and never pooled:

How we measured "this commit caused a bug"

SZZ, named for Śliwerski, Zimmermann and Zeller (When do changes induce fixes?, MSR 2005), finds the commits that fix bugs, runs git blame on the lines they changed, and labels the commit that last touched those lines as the one that introduced the bug. Run across a repository, it yields a labeled set of bug-inducing and clean commits.

We ran SZZ within each repository, comparing agent commits with human commits in the same codebase, which removes the confound that some projects are simply buggier. We then fit a logistic model with lines added, lines deleted and files touched as controls, plus a repository fixed effect, so every result reads as bug risk beyond what the size of the change explains. Size is the strongest single predictor of a bug-inducing commit, and without that control you mostly measure whether agents write bigger or smaller diffs.

Naive SZZ also carries a bias in favor of agents. It excludes fix commits from being counted as bug-inducers, and agents do a disproportionate share of fixing, so the plain method quietly shields them. We ran a stricter variant, B-SZZ, as a mandatory sensitivity check on every result. A finding that appears only under the friendly variant is not reported as real.

Result 1: agent commits are not more bug-inducing

These are the adjusted odds that a commit introduces a bug, by authorship tier, relative to human commits in the same repository. Below 1.0 means fewer bugs than the human baseline.

Every tier lands at or below the human line:

The T2 interval sits entirely below 1.0. T3 is indistinguishable from human, as expected for commits that are mostly human-authored.

Under strict B-SZZ the T2 effect attenuates toward the null: 0.79 with an interval of 0.68 to 1.01, which just touches 1.0. We therefore do not claim agents are "43% less bug-prone." The claim that holds across every variant and tier is narrower: there is no evidence that agent commits introduce more bugs than human commits in the same repository, and the point estimates are consistently protective.

Result 2: agent lines last longer

Durability is a second lens. "Agent code gets thrown away anyway" is a common objection even from people who accept the bug numbers, so we measured how much of it survives.

We sampled files across the repositories, took the lines added six or more months before the current HEAD, and computed the fraction still present. Comparisons were paired within each repository, so agent and human lines come from the same project and era.

In every repository with enough human-driven-agent volume to measure, agent lines survived at a higher rate than human lines: 17.9 percentage points in aggregate, with a confidence interval well clear of zero. The one exception anywhere in the study was a single bot-agent (T1) deployment that re-fixes its own files about three times as often as the others, and its lines churn out faster. Deployment pattern matters more than the agent's brand name.

Why this might hold

Three mechanisms are plausible, and they are not mutually exclusive.

Selection by review. These are merged commits, so each already cleared the project's review bar. Agent PRs may be scrutinized harder than a trusted teammate's, and part of the protective effect may measure reviewer stringency rather than raw model output. This is the caveat we weigh most.

Agents do maintenance. Agent commits skew toward fixes over net-new features. Small, localized changes to code that already works carry less risk than greenfield feature commits.

The human baseline is a realistic mix. It is not a senior engineer at their best; it includes rushed Fridays and unfamiliar subsystems. A consistent agent can clear that bar.

Limits of the study

Read the result with these in mind.

• Reviewed, merged commits only. The study says nothing about unreviewed agent output pushed straight to a branch.
• Only detected agent code is visible. Agent output committed by a developer under their own name with no marker counts as human, so every agent number is a floor.
• Agent-heavy repositories are young. The agent era is about twelve months deep, which is why every comparison is within-repo or against matched controls, never a raw pooled rate.
• The corpus is not a random sample. It skews toward projects that keep attribution and toward AI-adjacent tooling. We oversampled conventional non-AI projects as counterweight (a 17-year-old Java framework, a CMS, observability and chat platforms), but we make no population-wide claim.
• SZZ has known limits with tangled commits and large refactors. B-SZZ rides along with every number for that reason.

A finding for defect-prediction work

In repositories where agents write most commits, the cheap bug labels, keyword matching and "this file was touched by a fix," stop working. In the most extreme repository, 86% of files counted as buggy, which pushes any predictor toward a coin flip. It looked as if agent repositories break defect prediction.

The labeling shortcut breaks, not the underlying structure. With commit-level SZZ labels and the obvious self-fix churn removed, prediction recovered, and the usual signals (prior fixes, churn, ownership) work on agent code. That is why the repowise code-health score is built on commit-level history. The posts on why process metrics beat structural metrics and whether the health score predicts bugs cover the leakage traps in detail.

What it means for a team

A competent human plus an agent, with the human still reading the diff, produces commits that hold up at least as well as that human working alone. Keep review in the loop, watch bot deployments that re-fix their own files, and track bug-inducing history per author tier on your own repository rather than trusting either side's prior.

Reproduce it

The full report, per-repository numbers, strict-variant sensitivity tables and every analysis script are in the repowise benchmark repo. To see history-based health signals computed on a real codebase, open the public repo explorer. We will re-run the study when the agent era is eighteen months deep and publish the numbers either way.