OpenAI shifts the AI cost metric to successful tasks | Cheapest tokens can cost more per outcome
TL;DR
OpenAI CFO Sarah Friar proposes measuring AI by full cost per successful task, not the lowest token price.
OpenAI CFO Sarah Friar on July 17 proposed a new scorecard, Useful Intelligence per Dollar, that shifts AI economics from token prices to completed work. Companies would track four dimensions: useful work, cost per successful task, result dependability, and whether each dollar produces more value as usage scales.
Full cost includes compute, employee time, human review, retries, and rework, divided by tasks that clear the quality bar. A cheap model can cost more after repeated attempts than a higher-priced model that finishes in one pass. The lowest token price therefore does not always produce the lowest outcome cost. AI procurement ROI moves to deliverable results.
Friar cited Artificial Analysis Coding Agent Index v1.1. With different agent harnesses, GPT-5.6 Sol max in Codex scored 80.0, while Claude Fable 5 max in Claude Code scored 77.2. Aggregate output-token totals were 17,610,095 versus 38,395,071, 54% lower for the former. The index covers 321 tasks with three repeats per component and compares model-and-harness variants.
OpenAI's separate DeepSWE chart shows 72.7% versus 69.9% and a 36.2% reduction in estimated API cost. It does not present the same measure as the 54% Coding Agent Index output-token comparison. Artificial Analysis says missing telemetry is excluded rather than counted as zero. The raw totals are 17,610,095 versus 38,395,071.
via Artificial Analysis / OpenAI / Artificial Analysis methodology
Full cost includes compute, employee time, human review, retries, and rework, divided by tasks that clear the quality bar. A cheap model can cost more after repeated attempts than a higher-priced model that finishes in one pass. The lowest token price therefore does not always produce the lowest outcome cost. AI procurement ROI moves to deliverable results.
Friar cited Artificial Analysis Coding Agent Index v1.1. With different agent harnesses, GPT-5.6 Sol max in Codex scored 80.0, while Claude Fable 5 max in Claude Code scored 77.2. Aggregate output-token totals were 17,610,095 versus 38,395,071, 54% lower for the former. The index covers 321 tasks with three repeats per component and compares model-and-harness variants.
OpenAI's separate DeepSWE chart shows 72.7% versus 69.9% and a 36.2% reduction in estimated API cost. It does not present the same measure as the 54% Coding Agent Index output-token comparison. Artificial Analysis says missing telemetry is excluded rather than counted as zero. The raw totals are 17,610,095 versus 38,395,071.
via Artificial Analysis / OpenAI / Artificial Analysis methodology
