Pareto charts
Drag to zoom, double-click to reset.
Tap a point for details.
Ranking
How to read the scores
Agreement with a teacher
Gold is the mean of three samples from a teacher of roughly 4B-class capability, so a score measures agreement with that teacher, not correctness. The card puts the teacher's agreement with itself at 0.735; well above that line, a model is learning the teacher's quirks.
Two tables, not one
General models answer any question schema at request time and never saw these workflows.
Models fitted on train know the four workflows. The gap is the price of generality,
not a quality ranking, so “Both” draws fitted models as hollow diamonds.
Latency and price
Latency is p50 as reported: API round trips and local runs on different GPUs sit on one axis. The hover card names the hardware. Price is per million input tokens for hosted rows.
Pareto frontier
A model is on the frontier when no other visible model is at least as good on both axes and better on one. It is recomputed for every axis pair and filter.
Noise and ties
The ± in the hover card is a 95% interval over the 2,000 test decisions; “≈” and “within noise” mean the gap to the top of the table, or to Jev, is inside the 95% band for a difference. The decisions come in 400 cases of five, so they are not fully independent: a case bootstrap on the submissions that publish predictions put the effective sample at about 1,450–2,000, and because models answer the same cases, a paired test is narrower still. The bands shown are on the cautious side.