August 2026

Public benchmarks can't tell you which model to use. Your own traffic can.

We replayed 16 production jobs through a cheaper model and had a third-party model grade the results blind. Fourteen moved at ~91% lower cost; two didn't. The method, the bugs, and the harness we're open-sourcing.