Four benchmark suites

5.1 Prompts are fixed and published in full. Every prompt used by any suite lives in harness/prompts/, is version-controlled, and is printed or hashed into every report that used it. There is no private prompt.

5.2 The same prompt goes to every model. The only permitted differences are the mechanical requirements of an interface: where a system instruction is placed, how an image is encoded, the maximum output tokens, and whether a structured-output mode is used where the interface has one. Every such difference is listed in the report, per model.

5.3 No vendor-specific tuning. No per-model prompt rewriting, no per-model few-shot selection, no per-model temperature search, no retry with a different prompt after a poor answer, and no post-processing that only one system receives. Normalisation is one shared function applied identically to every system's output, and it is published with the harness.

5.4 Three runs per model per suite at identical settings. Reported as the mean with the standard deviation and the minimum and maximum. A single run is never published as a figure. Where three runs cannot be completed, the row says how many ran and why.

5.5 Every result is reproducible from a commit. Every report records the dataset version and content hash, the harness version and commit hash, the prompt set hash, the model version string as the provider reported it, the run date and time, the price list date, and the exact command line. A figure that cannot be reproduced from those is withdrawn, not defended.

5.6 Our own system is scored by the same harness, from the same commit, with the same prompts, on the same data, and appears in the same table. It gets no highlight, no bold, no annotation, no footnote and no position advantage: tables sort on the metric, not on the vendor. Where our number is worse, it stays where the sort puts it.

5.7 No sponsorship, no paid placement, no pre-agreed outcome. If a vendor ever funds the interface cost of running its own system, the report header says so, and it changes nothing else: the vendor sees its rows only through the pre-publication notice in section 8.3, at the same time as every other vendor.

5.8 A system that cannot complete a suite is reported as "not run" or "incomplete" with the reason, whether the reason is a missing key, a rate limit, a refusal, a context limit or a crash. A partial figure is never promoted into a headline table, and an incomplete row is never compared with a complete one.