Code Review at Agent Scale: Deciding Which Changes Get a Person

To review AI-generated pull requests at volume, stop treating every change as equal: score each one for risk and blast radius, send the risky ones to a person who owns that code, and let deterministic checks clear the rest. Reading every diff line by line works until merge volume outgrows the number of people who can read.

Anthropic put numbers on that curve. In its engineering post on redesigning source control (October 13, 2026, anthropic.com/engineering/redesigning-source-control), the company wrote: "In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day." That is a 12x increase in about five months. The post describes the internal version control system and review surface Anthropic built in response, called Scion. It is internal and, in the post's words, "by no means a suitable replacement for git for general use." The part that applies to every team is the volume.

Why does line-by-line review stop scaling?

Line-by-line review stops scaling because reviewer attention is fixed while the number of changes is not. A team that merges 40 changes a day can read all of them. A team whose agents open hundreds can read a sample, and the sample is chosen by whatever order the notifications arrive in.

The failure is quiet. Nobody decides to skip review. Every change gets a reviewer, and the change that touches a payment path looks identical in the queue to a typo fix. In our sample of merged pull requests from 10 large open-source repositories, reviewers asked for changes on 6.0% of 400 to 999 line PRs but on 0.9% of PRs over 1,000 lines, many of them mechanical (see PR size: how big should a pull request be). Volume pushes in the same direction.

People stay accountable for what their agents merge. That does not change when volume grows. What changes is how a person finds the changes that need their judgment. That is a routing problem, and routing can be automated.

What should you automate before a person looks?

Automate the questions a reviewer would otherwise answer by hand and slowly: how risky is this change, what else does it reach, what usually changes with it, which tests cover it, and who owns the code. Each has a deterministic answer from the repository's own history and dependency graph.

These are facts about where the diff lands, not opinions about the diff. A reviewer who sees "this touches a file with 25 co-change partners and five direct dependents" reads the change differently, and a team lead can set a rule on it.

What does this look like on a real change?

It looks like a short report that ranks the change and names what to look at. We indexed Flask (pallets/flask, shallow clone, 2026-10-29) in a scratch directory and scored the range that added the app.query route decorator and extended MethodView to support it:

repowise init --yes --no-prose --no-editor-setup
repowise risk 3596b1a..2a8a38b

Size and spread: 67th percentile of recent commits (diff size, not a bug predictor)

...
  +15 / -3 lines · 4 files · 4 dirs · 3 subsystems · entropy 1.63 · author exp 478
+----------------------------------------------------+
| File                         | Lines | Prior fixes |
|------------------------------+-------+-------------|
| CHANGES.rst                  |     1 |         5.0 |
| tests/test_basic.py          |     7 |         2.8 |
| src/flask/views.py           |     2 |         0.3 |
| src/flask/sansio/scaffold.py |     8 |         0.2 |
+----------------------------------------------------+
...

The change is small and mid-ranked. The diff-size score is 4.0/10. Percentile and score describe size and spread only, and the output says so. The second view, in PR mode, answers the blast-radius questions:

repowise risk -t src/flask/sansio/scaffold.py \
  --changed-file CHANGES.rst --changed-file src/flask/sansio/scaffold.py \
  --changed-file src/flask/views.py --changed-file tests/test_basic.py

...
  May break src/flask/blueprints.py, src/flask/debughelpers.py, src/flask/sansio/app.py, src/flask/sansio/blueprints.py, src/flask/templating.py
  Missing co-changes src/flask/sansio/blueprints.py, src/flask/sansio/app.py, src/flask/sessions.py
...
src/flask/sansio/scaffold.py
...
  health 7.6/10
...

The full output opens with a directive line: the PR touches 4 files, about 26 downstream files may be affected, and 68 historical co-changers are missing. Test gaps are withheld because no coverage report is connected, and 27 tests are listed as inferred.

Eight added lines, and the report still names about 26 downstream files, three files that historically change with scaffold.py and are not in the diff, and the file's ownership and dependents. A reviewer would not get that from the diff view. It does not say the change is wrong. It says where a person should look first, which is the part that scales.

With no coverage report connected, repowise withholds test gaps and lists the tests it infers from the import graph. That is an estimate, labelled as one.

How do you route risky changes to the right person?

Route on hotspot and ownership, not on who is next in the rotation. Churn-heavy files with a dominant owner should reach that owner (in scaffold.py, one maintainer owns 54% of commits), and a file with a bus factor of 9 across 18 contributors needs a different rule than one with a bus factor of 1.

A workable policy has three lanes:

• Auto-clear: low risk, no missing co-changes, tests reach the change, no new health findings. A person does not need to read it before merge.
• One reviewer: moderate risk or a hotspot file. The owner reads the diff with the signals open.
• Deep review: high risk, breaking callers, or a missing co-change that looks real. Two people, or a person plus a model pass.

Our pull request checklist lists what a reviewer checks once a change reaches them, and change risk explains how the score is built.

Where does a CI gate fit?

A CI gate fits as the backstop: it holds the line when nobody is watching. Two checks cover most of it. repowise risk can fail a build above a percentile of recent commits (--fail-above-percentile), and repowise health check origin/main...HEAD fails when the change adds complex, deeply nested, oversized or duplicated code. Both are deterministic, so a failure is reproducible and arguable, which matters when the author is an agent and the owner is a person who has to explain the block.

Use the gate for the rules you would otherwise enforce by memory, and keep thresholds loose enough that a red build means something.

How does the repowise PR bot do this on every pull request?

The repowise bot runs in two modes. Signals is free, deterministic, uses no model, and stays quiet unless a threshold is crossed. It posts the change risk, breaking callers and missing co-changes above. The tests signal is an estimate from the import graph unless you upload a coverage report, in which case a "Tests for this change" thread posts from measured coverage.

AI review is optional and on the Pro plan. It runs a model pass on every PR that changes code, with a deep review when risk calls for it. Review depth is light, auto or deep, and the default is auto. Cost averages about 9 cents a pull request. Commands in a PR comment: /repowise recheck and /repowise review deep.

Each comment has a "View on repowise" button that opens a public page for that PR with no sign-in (private repos need sign-in with repo access). Here is one: repowise-dev/repowise#4506. A "repowise review" check run is on by default and never fails the build unless you turn the health gate on. For how it compares with other tools, see best AI code review tools.

What about the volume problem itself?

Triage lowers the cost of each change. It does not change how many arrive. Two habits help upstream: keep changes small and single-purpose so the signals mean something, and make agents attach their own context (what they changed and why) so the person reading has the intent.

Scion is one company's answer at one extreme of volume. The idea underneath it is general: when merge counts climb past what reading can cover, the review process has to decide where attention goes. A team can start that today with git and a CI check.