Measured,
not claimed.

File coverage on 42 ContextBench instances pinned before any of this work started and kept sealed until the final measurement. Grading is deterministic and no LLM judge is involved.

Scroll the table sideways to see every column.

n = 42 sealed instances (41 for cocoindex, measured 2026-08-09 on the same instances and grading; one instance never answered and is excluded, not scored zero). Head to head against CodeGraph, get_answer wins 19, loses 1, ties 22, sign test p = 0.00004. The pooled 112-instance figure would be 0.835 and is deliberately not quoted: averaging the halves loses the only number that matters, which is how the tool does on instances it has never seen.

Every question in django's question set, six arms, byte-identical prompts, each tool's full advertised surface, and a freshly built index on the same pinned commit. Every tool was called on every question, so this is like for like.

Scroll the table sideways to see every column.

Codex (gpt-5.6-sol) on django/django at one pinned commit. Correcting for testing five tools at once, three reductions are solid and two are marginal. 5 of the 48 questions are missing from every arm equally because the run hit an API usage cap, so the figures are over the 43 questions all six arms completed; because all arms lost the same 5, the paired comparisons are unaffected.

The same six arms, the same servers and the same indexes, on a different agent harness. The result depends on the harness as much as on the tools.

Scroll the table sideways to see every column.

Claude Code (claude-sonnet-5), 15 questions. Treat the "agent used it" column as unstable, including our own 15 of 15: rerunning the same setup on later days returned 4 of 15 and then 3 of 15 for us, and 2 of 14 for CodeGraph. It is a property of the pairing of tool and harness on a given day, not of the tool. The same 15 questions on Opus called repowise on 7, inside the band fixed in advance as inconclusive, and that run's token column failed its own control, so no token figure is quoted from it. No quality or cost figure may be compared across the two harnesses, which were graded by different judges.

The same 15 questions and seed on qwen3:8b under Ollama via opencode, where inference is free and the saving shows up as time. Every cell called its tool.

Scroll the table sideways to see every column.

Two repowise rows, never combined. get_answer writes its answer with a hosted model, so the second row switches it off and leaves only tools that run against the local index; instructed directly to call get_answer, that agent could not reach it in any of its 15 cells. Only repowise and the control were run on this harness. The local-only row's answer quality is a null against the bare agent, so the claim is about as good an answer in roughly half the wall clock.

Precision / recall against call graphs generated by the Go and TypeScript toolchains. Both numbers must be read together.

Scroll the table sideways to see every column.

Precision / recall in every cell. Five tools, seven cells, 37,853 oracle edges. No tool that finds as much of the graph as Repowise gets more of it right in any cell. Repowise does not lead recall in the five Go cells and is most precise outright in only one cell.

Thirty stratified call edges per language per tool (forty for Java), read from source with imports and enclosing scope open.

Scroll the table sideways to see every column.

560 rows across nine languages; Java is read at 40 rows per side. Four language cells separate and five are ties. This comparison covers Repowise and CodeGraph only; the compiler-oracle table above covers the five-tool field on Go and TypeScript.

Both tools scored the same 2,770 files at the same leakage-free commit against the same defect labels. Paired tests throughout.

Scroll the table sideways to see every column.

CodeScene's precision lead is not statistically significant, but the behaviour behind it is worth having: it flags about 27 files where we flag 132. That is a more conservative threshold, trading recall for a short list a team will actually work through, and it is a deliberate design choice, not a weaker model. Our AUC edge is marginal, not significant at 0.05.

Graph construction only across the same 35 repositories. Three timed runs per cell after a discarded warmup; 175 cells and no failures.

Scroll the table sideways to see every column.

Memory is a clean sweep: Repowise is lowest on 35 of 35 repositories. Speed is level, not a win; CodeGraph is fastest on 16 repositories and Repowise on 14. Repowise builds this arm in memory without serialization, while CodeGraph writes SQLite.

We are the slowest indexer in the field, on every repo we measured, and it is not close. Measured on django/django.

Scroll the table sideways to see every column.

22x CodeGraph like for like, and 135x with prose on, which is what a default repowise init actually costs you. Both numbers ship. The reason is not an optimisation we forgot: in that same run repowise built 36,485 graph nodes, mined history across 2,630 files, rendered and embedded 3,392 documentation pages, extracted decision records and produced 5,317 health findings. It is also a one-time cost, since updates after the first index are incremental.