Model API pricing: compare the workload, not only token rates

A workload-cost model for comparing model APIs across input, output, cache, batch, tools, retries, latency, and successful task rate.

Model API cost depends on the entire workload: input, output, cached context, batch eligibility, tool calls, retries, latency, and success rate. Compare providers with the same prompt distribution and quality threshold, then calculate cost per successful task—not merely the listed price per million tokens.

What this guide covers

the fields required for a comparable rate card;
a workload formula and sensitivity model;
performance and reliability costs that the rate card omits;
pricing changes worth monitoring.

EXVIV market snapshot, August 4, 2026: the model APIs and inference category contained 120 companies and 46 reviewed events. Positioning accounted for 22 reviewed events and packaging for 7. The counts show where published source changes appeared; they do not measure usage, revenue, or technical performance.

Build the rate-card record

Official pages from OpenAI, Anthropic, Google Gemini, and Fireworks AI expose different models, tiers, and options. Record each comparable configuration separately.

Fields:

model/version and lifecycle;
input token rate;
output token rate;
cached read and write rates;
batch or flex rate;
long-context tier or threshold;
image, audio, search, tool, or reasoning charges;
fine-tuning/training and storage, when relevant;
regional or data-processing option;
quota, throughput, and service tier;
effective date and source.

Do not compare a preview model with a production SLA as if the commercial boundary were equal.

Define the workload distribution

An average prompt is not enough. Model at least three bands:

Band	Share	Input tokens	Output tokens	Cache hit	Tools	Retry rate
Small
Typical
Large

Add requests per month, concurrency, latency target, and batchable share. A provider can be economical for short synchronous calls and expensive for long-context agent work—or the reverse.

Calculate direct model cost

For one request band:

uncached_input_cost = uncached_input_tokens × input_rate
cache_read_cost     = cached_input_tokens × cache_read_rate
cache_write_cost    = cache_write_tokens × cache_write_rate
output_cost         = output_tokens × output_rate
tool_cost           = searches + tool-specific units

attempt_cost = input + cache + output + tool costs

Then include retries and success:

monthly_attempt_cost = attempts × attempt_cost
cost_per_success = monthly_attempt_cost ÷ successful tasks

A lower attempt cost can lose when the workflow needs more retries, longer prompts, or expensive post-processing to reach the same quality threshold.

Add system costs

Latency

Slow responses can reduce completion, consume worker time, or make an interactive product unusable. Model the value of latency only when the use case is latency-sensitive; do not invent a universal dollar penalty.

Reliability and fallback

Record timeout, error, rate-limit, and fallback frequency. Fallback may duplicate input, increase engineering complexity, or produce a different quality profile.

Context and retrieval

A cheaper model with a smaller effective context or weaker retrieval behavior may require chunking, summarization, vector search, or multiple calls. Those are part of the workload.

Evaluation and review

Quality must be measured on the same task set. Include evaluator/model cost and human review where material. A rate-card comparison without a quality boundary is incomplete.

Data and governance

Regional processing, retention, abuse monitoring, enterprise controls, and contractual terms can determine the eligible service before price.

Sensitivity table

Test the variables most likely to move the result:

Variable	Low	Expected	High
Monthly tasks
Input/task
Output/task
Cache hit rate
Retry rate
Success rate
Batchable share
Tool calls/task

The crossover point is often more useful than a single total. Ask when Provider A becomes cheaper than Provider B and whether that scenario matches real traffic.

Monitor more than price

Important movements include:

model launch or retirement;
input/output/cache rate changes;
context-tier boundary changes;
batch/flex availability;
regional processing surcharge or option;
quota and throughput changes;
tool/search pricing;
free-credit and trial changes;
enterprise data or retention terms.

A deprecation can matter more than a price cut when it forces evaluation and migration.

Decision worksheet

Workload and quality threshold:
As of:

Provider/model:
- input/output/cache rates:
- batch/flex:
- tools:
- region/tier:

Workload:
- monthly tasks:
- token distribution:
- cache behavior:
- retries/fallback:
- success rate:

Cost:
- direct attempts:
- eval/review:
- supporting infrastructure:
- cost per successful task:

Constraints/unknowns:
Migration trigger:

Use EXVIV's model APIs and inference market, the company directory, and Evidence Compare to inspect current source records. Keep your workload calculator separate from the vendor comparison so rate changes can be updated without rebuilding the decision logic.

Sources and further reading

Method note

Model rate cards and model versions are highly volatile. Recheck every cited rate and lifecycle condition within 30 days and benchmark the same representative task set. This article provides the cost model, not a quote or quality ranking.