BenchCAD 1.0 · Projects

The live benchmark — single manufacturable parts reconstructed from multi-view renders, graded by executing the program and comparing geometry. IoU-score = voxel IoU × exec%, re-graded from raw predictions, never self-reported.

17,900 execution-verified CadQuery parts across 106 industrial families — about half anchored to real ISO / DIN / EN / ASME / IEC specification tables (47 codes).
Four matched tasks on the same parts: Vision2Code (the flagship), Vision QA, Code QA, Code Edit.
Deterministic grading — geometry by voxel IoU, QA by ratio accuracy. No LLM judge.

Contribute a part family

BenchCAD grows by family — a parametric CadQuery generator for a real industrial part, shipped with QA and edit pairs:

Start from CONTRIBUTING.md and the example in contributions/<family>/ (family.json · generator.py · qa/ · edits/).
Validate locally with tools/validate_task.py and tools/validate_family.py — no API keys needed.
Maintainers re-check and ingest accepted families into the HuggingFace dataset — every accepted family is credited on the contributors wall.

Running a model instead? See submit a model to the leaderboard.