What changed
| 2.0 | 2.1 | |
|---|---|---|
| Rounds per case | 30 | 60 |
| Output tokens per case | no budget | 500,000, thinking included |
| Reasoning across rounds | dropped | carried |
| The prompt states the limits | no | yes, with rounds and tokens spent so far |
| The prompt says no submit scores 0 | no | yes |
| A discarded, re-asked reply counts against the budget | — | no |
These limits are for the standard 2.1 harness. The customized-harness track (Claude Code, Codex) has no token or time limit.
Scores from 2.1 are not comparable with 2.0.
Task 6 also matches components more strictly: a prediction can only claim the part nearest to it, and two components answered at the same spot both count as wrong (how Task 6 is scored).
The case files are unchanged, and 2.0 still reproduces from its own tag; details are in the release notes.
Results
Credit to Anthropic for the Claude results. With the 2.1 harness, 5 passes per model, Claude Opus 5.5 leads the four models in that run from medium effort up, with 0.49 at max. In Claude Code, Opus 5.5 reaches 0.65. Both boards are on the leaderboard.
Added 2026-11-09: Claude Fable 5.2, from Anthropic's system card (§8.13.2). At max effort it scores 0.754 with Anthropic's own run of the 2.1 protocol and 0.835 with tools. The lower efforts in the charts are read off the card's chart (≈).
Added 2026-11-17: OpenAI's post Introducing GPT-6.2 Astra and Sol reports BenchCAD 2.1 Core at every effort. At max, GPT-6.2 Astra scores 0.825 for $5.43 per case, GPT-6.2 Sol 0.731 for $1.60, GPT-6 Astra 0.626 and GPT-6.1 Sol 0.526. Run in Codex, credit to OpenAI; they're on the customized-harness board (Figure 1, second tab).
Acknowledgments
We thank Anthropic for the Claude results, and OpenAI for the GPT results.
Citation
2.1 is a new release of the same benchmark, so please cite BenchCAD 2.0, and say in your paper that you used release 2.1 (tag v2.1):
@misc{benchcad2_2026,
title = {BenchCAD 2.0: Benchmarking Programmatic CAD on Real Engineering Work},
author = {{BenchCAD Team}},
year = {2026},
howpublished = {\url{https://benchcad.com/blog/benchcad-2.html}},
note = {Blog post}
}
@misc{benchcad2026,
title = {BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD},
author = {Zhang, Haozhe and Liu, Kaichen and Chen, Miaomiao and Li, Lei
and Yang, Shaojie and Peng, Cheng and Chen, Hanjie},
year = {2026},
eprint = {2605.10865},
archivePrefix= {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2605.10865}
}