OpenAI reports BenchCAD 2.1 for GPT-6.2 Astra and Sol
OpenAI's launch post for GPT-6.2 Astra and Sol reports BenchCAD 2.1 Core at every reasoning effort: GPT-6.2 Astra scores 0.825 at max for $5.43 per case and GPT-6.2 Sol 0.731 for $1.60, up from 0.626 for GPT-6 Astra. Run in Codex, credit to OpenAI; the results are on the customized-harness board.
Anthropic: Claude can design electronic prototypes from start to finish
In a new research post, Anthropic and Diode Computers show Claude Fable 5.2 designing three working electronic prototypes, a speakerphone, a voice-assistant hub and a handheld pet, from concept to manufactured boards. The post cites BenchCAD 2.1, where Fable 5.2 scores 0.835 with tools, nearly double Claude Fable 5.1.
Claude Fable 5.2 on BenchCAD 2.1: 0.754 with the standard harness, 0.835 with tools
Anthropic's Fable 5.2 system card reports BenchCAD 2.1 Core: at max effort, Claude Fable 5.2 scores 0.754 with the standard 2.1 harness and 0.835 with tools, ahead of Claude Opus 5.5 at 0.490 and 0.653. Congratulations to the Anthropic team!
By reasoning effort, with tools, Fable 5.2 matches Opus 5.5 at low cost and pulls ahead from high effort up. At max effort, with and without tools:
BenchCAD 2.1: a new standard harness for Core, and a track for customized harnesses
BenchCAD 2.1 gives each Core case 60 rounds and 500,000 output tokens, thinking included, and carries the model's reasoning from round to round. A new track runs Core in a customized harness, such as a lab's own agent, on a board of its own. Anthropic's runs of both are on the leaderboard.
OpenAI on BenchCAD 2.0: a newer internal model scores 0.875
OpenAI's post on AI for physical engineering measures progress with BenchCAD 2.0: a newer internal model averages 0.875, against 0.594 for GPT-6 Astra at its highest reasoning setting, the best score on Core at our launch. Congratulations to the OpenAI team!
In the chart, the internal model's cheapest setting already scores about 0.76, above Astra's best at about the same cost.
Kimi K3.1 Pro on BenchCAD: 0.927 with tools, 0.728 without
Moonshot's K3.1 Pro launch post reports 0.927 with tools and 0.728 without on a random 1,000-file Vision2Code subset at max effort, first among open models on our leaderboard in both settings and up from 0.367 for Kimi K3 (our re-grade on the full set). Congratulations to the Kimi team!
Introducing BenchCAD 2.0: the best model scores 0.59 on Core
BenchCAD 2.0 tests professional hardware engineering work: parts and assemblies from engineering drawings or rendered views, and board wiring from renders. Five frontier models ran on Core, 100 held-out cases, at every reasoning effort; the best, GPT-6 Astra at max effort, scores 0.59.
Claude Haiku 5.5 on BenchCAD: 0.870 with tools, 0.670 without
Anthropic's Haiku 5.5 system card reports 0.870 with tools and 0.670 without on the 1,000-file Vision2Code subset, up from 0.210 and 0.155 for Haiku 4.5. Congratulations to the Anthropic team!
BenchCAD Vision2Code ranking on 2026-10-07
With tools, Haiku 5.5 reaches about 0.79 at roughly $0.25 a task, ahead of every other model in the card at that cost, and 0.870 at under $2.
Claude Sonnet 5.5 on BenchCAD: 0.963 with tools, 0.747 without
Anthropic's Sonnet 5.5 system card reports 0.963 with tools and 0.747 without on the 1,000-file Vision2Code subset: first in both settings on our leaderboard, just ahead of Opus 5.5 (0.962 / 0.730) and GPT-6 Astra (0.959 with tools). Congratulations to the Anthropic team!
BenchCAD Vision2Code ranking on 2026-09-28
As in Opus 5.5's card, the score climbs with test-time compute, most steeply with tools: Sonnet 5.5 goes from about 0.33 at under half a cent a task to 0.963 at about $6.
Claude Opus 5.5 on BenchCAD: 0.962 with tools, 0.730 without
Anthropic's Opus 5.5 system card reports 0.730 without tools and 0.962 with tools on the 1,000-file Vision2Code subset, the highest score published in either setting.
BenchCAD Vision2Code ranking on 2026-09-22
The same card re-runs Anthropic's earlier models with each view rendered at 256 px, as the reference harness does; earlier cards had used 128 px views. Fable 5.1 is now 0.606 / 0.926 (was 0.437 / 0.843), Opus 5 0.497 / 0.899 (was 0.366 / 0.821) and Sonnet 5 0.322 / 0.519.
OpenAI's GPT-6 Astra launch lists BenchCAD at 0.959 with a Python tool on the 1,000-file Vision2Code subset — the second lead change in three days, after Claude Fable 5.1's 0.843 on September 1.
BenchCAD Vision2Code ranking on 2026-09-03
With tools
Two caveats we are keeping visible. OpenAI has published no no-tools figure, so Astra carries an agentic score only and takes no rank in the plain column. And OpenAI notes its own Claude comparison runs “used modified evaluation settings” — the Claude numbers on our leaderboard come from Anthropic's system cards, not from OpenAI's table.
Claude Fable 5.1 on BenchCAD: 0.843 with tools, 0.437 without
Anthropic's Fable 5.1 system card reports it on BenchCAD's Vision2Code subset — 0.437 without tools and 0.843 with, the highest with-tools score any lab has published, narrowly past GPT-5.6 Sol's 0.834.
BenchCAD Vision2Code ranking on 2026-09-01
Without tools the order is unchanged — Sol still leads 0.706 to 0.437. The whole gain sits in the agentic setting, and it keeps scaling with test-time compute: with tools, Fable 5.1 nearly doubles its own no-tools score.
Claude Opus 5 on BenchCAD: 0.821 with tools, 0.366 without
Anthropic's Opus 5 system card reports it on BenchCAD's Vision2Code subset — 0.366 without tools, 0.821 with — and it upstreams two fixes to our reference (a camera-position typo and raw-shape grading).
BenchCAD Vision2Code ranking on 2026-07-24
The story isn't the single number — it's the slope. On BenchCAD, Claude's scores now scale steeply with test-time compute, especially once the model has tools to crop and verify its own renders. More effort buys markedly more accuracy.
The card also re-runs Opus 4.8 (0.277 / 0.521), Mythos 5 (0.378 / 0.678 on the subset, where its own card had 0.379 / 0.650) and Sonnet 5 (0.267 / 0.376) on the same harness as Opus 5.
GPT-5.6 Sol on BenchCAD: 0.834 with tools, 0.706 without
OpenAI lists BenchCAD in its GPT-5.6 launch table, next to OSWorld and BrowseComp — the second frontier lab after Anthropic to report it in a flagship launch, and another sign AI for hardware is now a strategic front.
BenchCAD Vision2Code ranking on 2026-07-09
Scores exactly as printed in OpenAI's launch table (computer-use section, 2026-07-09), self-reported voxel IoU. The Claude bars are Anthropic's own published figures, republished by OpenAI. Hover any bar for its source and split.
Claude Sonnet 5 on BenchCAD: 0.373 with tools, 0.266 without
Anthropic's Sonnet 5 system card reports it on BenchCAD's Vision2Code subset — 0.266 without tools, 0.373 with — another flagship model card measuring on the benchmark.
BenchCAD Vision2Code ranking on 2026-06-30
The card also re-runs Opus 4.8 (0.280 / 0.535) and Mythos Preview (0.359 / 0.611): the Fable 5 / Mythos 5 card had applied only the camera-prompt fix, and these are the first scores with the raw-shape grading fix in place too.
Claude Mythos 5 on BenchCAD: 0.650 with tools, 0.384 without
Anthropic's Claude Fable 5 / Mythos 5 system card evaluates on BenchCAD Vision2Code:
their strongest model reaches ~0.38 voxel IoU on the full set, and ~0.65
with Python tools on a 1,000-file subset. The first frontier lab to fold execution-grounded CAD
into how it measures its models — an early marker of what has since become a strategic front.