Coding Agent Value Lab

Independent analysis of coding models, harnesses, reasoning levels, costs, and subscription efficiency.

Decision lens

Benchmarks tell you who scored highest. This lab tells you what to use.

13 publishable configurations · refreshed 2026-07-13Updated 2026-07-13Evidence-first

This week in coding agents

What's Changed Since Last Week?

Baseline · 2026-07-13

Baseline established. The next research refresh will report material changes.

Dashboard

Measured research at a glance.

Measured leader

83.4%

GPT-5.5 · Codex CLI · interval 81.2–85.6%

Official verified

1 source runsha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9

Measured configurations

13

Across 1 benchmark version(s) · refreshed 2026-07-13.

First-party measured

13 source runssha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2
sha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090
sha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf
sha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Leader gap

0.3 pts

Compared with Claude 5 Fable · ranked by measured score.

Official verified

2 source runssha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442

Evidence health

13/13 verified

Accessible, non-warning runs with first-party or independently verified evidence.

First-party measured

13 source runssha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf
sha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2
sha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090
sha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef
sha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Recommendation

Measured leaderboard live; value ranking pending.

Verified benchmark results are published, but sourced per-run usage is still required for cost and value recommendations.

Measured configurations

13

Cost-ready configurations

0

Research controls

Ask the page a practical question.

The controls are part of the data contract now; the research pipeline can connect them to filtered recommendations as coverage grows.

SmallMediumLargeXlSubscription + APISubscriptionAPIBest valueBest qualityMost work per windowFastest

Today's insights

What the research layer is designed to surface.

Research boundary

Cost conclusions are waiting for sourced run usage.

The current benchmark publishes measured accuracy, but not the token usage required to calculate cost per evaluated task.

Measured comparison

Measured performance

Benchmark pass rate

Measured performance

Benchmark pass rate

GPT-5.5 · Codex CLI83.4%

Interval 81.2%–85.6%

1 source runsha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9

Claude 5 Fable · Claude Code83.1%

Interval 81.1%–85.1%

1 source runsha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442

Claude 5 Fable · Terminus 280.4%

Interval 78.1%–82.7%

1 source runsha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859

Claude Opus 4.8 · Claude Code78.9%

Interval 76.4%–81.4%

1 source runsha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2

GPT-5.5 · Terminus 278.2%

Interval 75.8%–80.6%

1 source runsha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048

Claude Opus 4.8 · Terminus 274.6%

Interval 72.2%–77.0%

1 source runsha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23

Gemini 3 Pro · Terminus 274.4%

Interval 71.8%–77.0%

1 source runsha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef

Gemini 3.1 Pro · Gemini CLI70.7%

Interval 67.8%–73.6%

1 source runsha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090

Gemini 3.1 Pro · Terminus 270.3%

Interval 67.4%–73.2%

1 source runsha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d

Claude Opus 4.7 · Claude Code69.7%

Interval 67.0%–72.4%

1 source runsha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf

Gemini 3 Pro · Gemini CLI66.3%

Interval 63.6%–69.0%

1 source runsha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1

Claude Opus 4.7 · Terminus 266.1%

Interval 63.4%–68.8%

1 source runsha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668

GLM 5.1 · Claude Code58.7%

Interval 56.3%–61.1%

1 source runsha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Measured comparison

Harness comparison

Claude 5 Fableterminal-bench@2.1

Terminus 2 80.4%Claude Code 83.1%Observed difference 2.7 pts

2 source runssha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442

Claude Opus 4.7terminal-bench@2.1

Terminus 2 66.1%Claude Code 69.7%Observed difference 3.6 pts

2 source runssha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf

Claude Opus 4.8terminal-bench@2.1

Terminus 2 74.6%Claude Code 78.9%Observed difference 4.3 pts

2 source runssha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2

Gemini 3 Proterminal-bench@2.1

Gemini CLI 66.3%Terminus 2 74.4%Observed difference 8.1 pts

2 source runssha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef

Gemini 3.1 Proterminal-bench@2.1

Terminus 2 70.3%Gemini CLI 70.7%Observed difference 0.4 pts

2 source runssha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090

GPT-5.5terminal-bench@2.1

Terminus 2 78.2%Codex CLI 83.4%Observed difference 5.2 pts

2 source runssha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9

Measured comparison

Research coverage

Collected13

13 source runssha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf
sha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2
sha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090
sha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef
sha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Measured13

13 source runssha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf
sha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2
sha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090
sha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef
sha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Cost-ready0

0 source runsNo source run IDs supplied.

Value-ready0

0 source runsNo source run IDs supplied.

Supporting evidence

Context for the measured results.

Top three

3

Ranked publishable configurations by measured benchmark score.

Official verified

3 source runssha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859

Largest harness difference

8.1 pts

Gemini 3 Pro · terminal-bench@2.1

First-party measured

2 source runssha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef

Research coverage

13/0 stages

Collected, measured, cost-ready, and value-ready evidence stages.

First-party measured

13 source runssha256:cedd10f727eac691532c607e117ff0d3b22725c19bf84d44062dd42e77c0afc9
sha256:581d154c3ed3efe3a744c0422bf2bfc549e7440306a43372dfce0e377dfe3442
sha256:b346bebd013bd4c1234d30e6d454cc76fe330e15e76caf0ca313c0fb3c0da859
sha256:ca1a8883459f2e1421e6d294f23ff7d07852050cda68fb69c4da2b6c977126e2
sha256:f31d0e544ba8274c2b631a0677a41e6a27f2e7f8c8a17f3b8c612c11f6d32048
sha256:ba002e0364afa78f1f3df03ddf80f7a10508d8c6d18f4d6651bfa8219a791b23
sha256:15709cda1b1ab71510ae25251dd9123f7009d613975ed3ab67bc598d0c23b5ef
sha256:5bbc4ec6dece6c48e7225afc52599e5c978c697e7cbec38b7d85d51d8571d090
sha256:7a4dcf8cc24f4ad8fe70ef1341204ff3a32742e68e01b3d34a87585facde148d
sha256:f3594bd33bf40bdca987c12c330401e9863857fc93368ad7269693351e2d5edf
sha256:f05d0b50c0fb8f5e44226bb5d3fc6f586b3002b359b3e70bda7282d1e41ebbb1
sha256:ab01d8ae5e8ec86b324ccf8a17a1043ea1895a0f54e1ba761d22b7706e34c668
sha256:9954753b7d8f8e31f8e4088a3f4fefa57b6f8d534b6573a186f079744428f008

Methodology

Useful ranges over false precision.

Benchmark versions are isolated; evidence, source dates, uncertainty, harness, and reasoning effort remain visible.

Assumptions

A two-point practical-equivalence band is used for the default value recommendation.

Raw data

Every published claim will retain its benchmark version, harness, reasoning level, evidence class, date, and source.

Data contract is versioned with the page

terminal-bench-2-1 terminal-bench-2-1-release terminal-bench-integrity