How to cut LLM API costs

What drives an LLM API bill?

Token volume and model choice drive most of it. OpenAI’s cost optimization guide boils lower cost and latency down to three strategies, which are fewer requests, fewer input and output tokens, and a smaller model. Anthropic prices every model per million input and output tokens, with separate rates for cache writes and cache reads (Anthropic prompt caching).

Per-model rates are on OpenAI API pricing and Anthropic API pricing.

Which cost levers do OpenAI and Anthropic offer?

Mostly the same ones. Both have smaller models, prompt caching, batch processing, output limits, structured outputs and spend controls, and OpenAI also documents flex processing. The table puts each lever next to the feature that provides it and the discount as each provider states it.

Lever	OpenAI feature	Anthropic feature	Discount as officially stated	When it fits
Right-size the model	”Select a smaller model” (cost optimization)	Start efficiency-first with Claude Haiku 5.5, or tune the effort parameter (choosing a model)	Depends on the models compared. See the pricing pages.	High-volume, simple tasks
Prompt caching	On by default for supported models (prompt caching)	cache_control, automatic or on chosen blocks (prompt caching)	OpenAI cached input “discounted up to 95%”. Anthropic cache reads at 0.1 times base input on most models.	Long prefixes sent again and again
Batch processing	Batch API (batch)	Message Batches API (batch processing)	OpenAI “50% cost discount”. Anthropic “50 percent of the standard API prices”.	Jobs that can wait for results
Flex processing	service_tier: "flex" (flex processing)	Not covered in the Anthropic docs reviewed	Priced at Batch API rates, per OpenAI	Low-priority work that can tolerate delays and retries
Shorter prompts and outputs	Minimize tokens (cost optimization guide), lower max_tokens	max_tokens on every request	No rate discount. Fewer tokens are billed.	Every workload
Structured outputs	JSON Schema responses (structured outputs)	Constrained decoding (structured outputs)	No rate discount. Fewer retried calls.	Apps that parse model output
Spend limits and alerts	Organization and project spend limits (spend limits)	Organization spend limit and workspace spend limits (rate limits, workspaces)	No rate discount. Caps monthly spend.	Any team with a budget

How much can a smaller model save?

The gap between the two models’ token rates, so it depends entirely on which pair you compare. OpenAI lists “select a smaller model” as a core strategy and says to use models that keep accuracy while cutting cost and latency (OpenAI cost optimization).

Anthropic describes two ways to start (choosing a model). Efficiency-first means building with Claude Haiku 5.5, testing, and upgrading only where you find a specific capability gap. Anthropic recommends it for cost-sensitive work and for high-volume, straightforward tasks. Capability-first means starting with Claude Opus 5.5 and moving to more efficient models later.

Before you switch at all, Anthropic says tuning the effort parameter on supported models “is often a better lever than switching models.” Whichever you do, run your own evaluation set before and after, so a cheaper model doesn’t quietly cost you quality.

How does prompt caching lower costs?

It bills a reused prompt prefix at a lower input rate. The two providers set it up and price it differently.

On OpenAI, caching is on by default for supported models, and cached input is “discounted up to 95%” (OpenAI prompt caching). The rules depend on the model. On GPT-5.6 and later:

• A prefix must reach 1,024 tokens to be cached.
• Cache writes cost 1.25 times the standard input rate for the default 30-minute lifetime, or 2.5 times for a 12-hour lifetime in the Responses API.
• Reads cost 0.1 times standard input on most of these models and 0.05 times on GPT-6.1 Sol and GPT-6.1 Terra.

Earlier models use implicit caching only, with no additional cache-write charge.

OpenAI’s cache matches the whole rendered prefix. Change the model, the tools, the structured output schema or other listed settings ahead of the cached point and the request won’t match.

On Anthropic, you add cache_control once at the top level of a request for automatic caching, or place it on individual content blocks (Anthropic prompt caching). Prices are multiples of the base input price. A cache write with the default 5-minute lifetime costs 1.25 times base input, and one with a 1-hour lifetime costs 2 times. Reads cost 0.1 times base input, and Anthropic lists lower read rates for some models in its pricing table. These multipliers stack with other pricing modifiers, the Batch API discount included.

With either provider, put stable content such as instructions, tool definitions and reference documents first, and put whatever changes on every request last. A timestamp or user message near the top breaks the match and turns every request into a cache write.

When should you use batch processing?

Whenever nobody is waiting on the answer. Evaluations, classification and bulk generation all fit, and both providers price batch work at half their standard rates.

OpenAI’s Batch API takes requests in a .jsonl file, with a “50% cost discount compared to synchronous APIs”, a separate pool of higher rate limits, and completion within 24 hours (OpenAI batch). Each input file holds requests for one model only.

Anthropic’s Message Batches charge all usage at 50 percent of standard API prices. Most batches finish in under an hour, and a batch expires if processing doesn’t finish within 24 hours. One batch holds up to 100,000 requests or 256 MB (Anthropic batch processing).

Anthropic suggests the 1-hour cache lifetime for batches that share context, because batches can take longer than 5 minutes. It also warns that batches can go slightly over a workspace spend limit because of their high throughput.

What is OpenAI flex processing?

An OpenAI service tier that trades speed and availability for a lower price. Set service_tier to flex on a Responses or Chat Completions request, and tokens are priced at Batch API rates, with additional discounts from prompt caching (OpenAI flex processing).

It comes with conditions. Flex is in beta with limited model availability, and the supported models are listed on OpenAI’s pricing page. Responses are slower, so OpenAI suggests raising the SDK timeout above its 10-minute default. When capacity runs short, requests return a 429 Resource Unavailable error, and OpenAI says you are not charged for those.

Flex fits evaluations, data enrichment and other asynchronous work where you’d rather make a regular API call than build a batch file.

How do shorter prompts and outputs reduce the bill?

Every token you send and every token the model generates is billed, so fewer tokens cost less on any model. OpenAI’s guidance is to cut input tokens and aim for shorter outputs (OpenAI cost optimization).

Start with what you send. Repeated instructions, unused tool definitions and old conversation turns you no longer need are paid for on every call. Then ask for the length you want in the prompt and set max_tokens as a ceiling. OpenAI notes that requests with a lower max_tokens also see less latency (production best practices). On OpenAI, watch n and best_of as well, since values above 1 generate several outputs per request.

On Anthropic, output rate limits count only the tokens actually generated, and max_tokens doesn’t factor in (Anthropic rate limits). The ceiling is there to stop runaway answers. It doesn’t reserve capacity.

Do structured outputs save money?

Yes, by cutting retries. If a response fails to parse and you send the request again, you pay for both calls.

OpenAI says Structured Outputs make the model always follow your JSON Schema, so there’s “no need to validate or retry incorrectly formatted responses” (OpenAI structured outputs). Anthropic says its structured outputs guarantee schema-compliant responses through constrained decoding, with “no retries needed for schema violations” (Anthropic structured outputs).

On OpenAI the output schema is part of the cached prefix. If you also rely on prompt caching, keep the schema stable (OpenAI prompt caching).

How do spend limits and alerts protect a budget?

Spend limits don’t lower any price, but they stop a bug or a traffic spike from running up a bill nobody planned for. Alerts notify you while traffic continues. Hard limits stop traffic.

On OpenAI (spend limits), you can set a monthly spend limit for the organization or for one project. Turning on “Enforce a hard limit” makes affected requests fail with a 429 error (organization_spend_limit_exceeded or project_spend_limit_exceeded). Enforcement isn’t instantaneous, so recorded spend can slightly exceed the limit. Spend alerts only send a notification and leave traffic running.

On Anthropic (rate limits, workspaces), each usage tier carries a monthly spend cap. When an organization reaches it, API usage pauses until 00:00 UTC on the first day of the next month unless it gets a higher limit sooner. You can set your own lower limit in the Spend limits section of the Billing page in the Claude Console. Spend limits and alerts can also be set per workspace on the Workspaces page. Anthropic notes you can’t set spend limits on the Default Workspace.

How should you monitor usage while you cut costs?

Check usage after each change, so you can see which lever actually moved the bill. OpenAI has a usage dashboard, and its prompt caching guide points to a Prompt Caching Dashboard that shows cache read hit rates. Anthropic’s Usage and Cost Admin API returns data similar to the Console’s Usage and Cost pages. Anthropic lists rate limit optimization, including tuning prompt caching, among its uses (Anthropic Usage and Cost API).

Dashboards, cost APIs, per-project keys and receipts are covered in how to track AI API spend.

If OpenAI is your main bill, read how to cut OpenAI API costs on the ernd blog. To compare models at list prices, try the AI API cost calculator.