AI Picker
Cost · Research Note

Before You Scale with CostPerPrompt: The Real API Cost of Pricing Calculators

You are deciding whether to rely on an API pricing calculator — CostPerPrompt specifically, but the logic applies to any tool in this category — to plan, forecast, or control AI API spend before scaling usage.

AI Picker rule: Draft preserves the core judgment, all internal links, caveats, and source boundaries; no forbidden topics or unverified factual claims detected.

The decision

You are deciding whether to rely on an API pricing calculator — CostPerPrompt specifically, but the logic applies to any tool in this category — to plan, forecast, or control AI API spend before scaling usage.

Use it to build your first cost model, not to replace your invoice. Calculators encode assumptions about token counts, prompt shapes, retry rates, and routing behavior that almost never match your real workload. The risk is not that the calculator is wrong. The risk is that a specific number from a calculator feels like a verified number, and teams stop checking.

Core judgment

Use a pricing calculator to build your first cost model. Do not use it to replace your invoice.

The specific tool — CostPerPrompt or any calculator in this category — is not the issue. The issue is structural: calculators can verify unit prices, which are public and simple. They cannot verify your workload, which is private, messy, and specific to your harness. A calculator output looks like a measurement. It is an estimate wearing a measurement's clothes. Teams that treat it as a measurement stop checking, and the first sign they stopped checking is a surprise bill.

Why this is on the desk now

CostPerPrompt appeared as a Show HN post describing itself as a live AI API pricing and real-workload cost calculator. Community signal is modest — 18 stars, 7 comments at time of writing — so there is not yet enough adoption data to assess whether its estimates hold up across diverse production workloads.

What makes this worth a field note is not the specific tool. It is the category problem. Pricing calculators are proliferating, and they all share the same structural gap: they show you unit prices (usually accurate and public) while estimating workload costs using assumptions that are invisible, unverified, and almost always wrong in the same direction — undercount.

Concrete scenario

A team planning to deploy a coding agent runs their expected usage through a pricing calculator. They enter: model name, expected prompts per day, average prompt length. The calculator returns a monthly figure. The number looks reasonable. They approve the budget.

Then production starts. The invoice is 3–5× the estimate. Why:

  • The calculator didn't know about token overhead. Their agent's harness injects system prompts, tool-call scaffolding, and retrieved context before the user's actual prompt touches the model. That scaffolding can add tens of thousands of tokens per call — tokens the user never typed but still pays for. See Evaluating Real API Costs: Claude Code vs OpenCode Token Overhead for why this is the variable calculators miss most.
  • The calculator modeled single calls, not sessions. In a coding session, context accumulates. Call 1 sends 5k tokens. Call 20 of the same session sends 80k tokens because the conversation history is re-sent each time. The calculator's "average prompt length" assumption flattens this curve into a constant.
  • The calculator didn't model retries. If a task succeeds 70% of the time and retries twice on failure, the real cost per successful output is roughly 2.4× the per-call estimate. No calculator knows your failure rate.
  • The calculator may not account for caching discounts. Provider-side prompt caching can cut costs significantly — but only for cacheable patterns. If the calculator uses uncached pricing and your workload qualifies for caching, it overestimates. If it assumes caching and yours doesn't qualify, it underestimates. Either way, the gap is invisible until the invoice arrives.

The team blames the calculator. But the calculator wasn't wrong about prices. It was wrong about workload — and workload is the part it cannot see.

Non-obvious insight

The part calculators get right is the part you don't need help with. The part you need help with is the part they can't verify.

Per-token prices are public, documented, and change infrequently. You can look them up in two minutes. The hard part of API cost forecasting is not knowing the price of a token — it is knowing how many tokens your real workload consumes. That number depends on your harness, your retry logic, your context growth pattern, your caching behavior, and your fallback routing. None of these are visible to a calculator.

This creates the real trap: the calculator's accuracy on unit prices (the easy part) lends credibility to its workload estimates (the hard part). A team sees that the per-token price matches the provider's pricing page, concludes the calculator is reliable, and extends that trust to the monthly estimate — which is built on assumptions the team never inspected.

Teams surprised by API bills are usually surprised for the same reason: they knew the price per token but not their own workload. A calculator does not fix this gap. It can deepen it, because "I ran it through a calculator" feels more responsible than "I guessed."

The practical rule

Run the calculator once to get a floor estimate. Then run a 24-hour real workload test with provider billing dashboards open. Compare the two numbers. The gap between them is your actual cost of abstraction.

The variables calculators most often get wrong or omit are the ones above — token overhead, retries, context growth, caching — plus one more:

  1. Rate-limit and quota behavior. Hitting a rate limit does not just pause you. It can force a fallback to a more expensive model, or kill a batch run mid-stream.

Checklist

Before trusting any API pricing calculator at scale:

  • [ ] Does the tool publish its workload assumptions (prompt length, tokens per request, retry model)?
  • [ ] Does it pull live pricing from provider APIs or maintain a manual table?
  • [ ] Have you run a 24-hour real-workload test and compared estimated vs. actual cost?
  • [ ] Does the calculator account for token overhead from your specific harness or framework?
  • [ ] Does it model context accumulation across multi-turn sessions?
  • [ ] Does it distinguish between cached and uncached token pricing?
  • [ ] Do you have a provider billing alert set independently of the calculator?
  • [ ] If the calculator is wrong by 2×, can your budget absorb it?

If you cannot check at least five of these, treat the calculator output as a guess, not a forecast.

Evidence to verify

Official sources to check:

  • costperprompt.com — confirm which providers, models, and pricing tiers it covers. Check whether it publishes its calculation methodology or workload assumptions.
  • Verify whether the tool pulls live pricing from provider APIs or maintains a manual price table. A manual table can go stale between provider updates.

Data gaps to flag:

  • GitHub stars (18) and HN comments (7) are adoption signals, not accuracy signals. They tell you the tool is early, not whether its estimates are reliable.
  • No benchmark data, no third-party cost-comparison study, no published accuracy audit was available at time of writing.

Market voice: The HN post generated limited discussion. Treat the existence of the post as evidence that developers want this kind of tool, not as evidence that this tool's estimates are correct.

When I would not trust this

  • When the calculator does not publish its methodology. A black-box estimate is no better than a gut feeling with a decimal point.
  • When the tool covers only headline per-token prices. If it cannot model retries, context growth, or caching, its estimate will undercount real spend — and undercounting is more dangerous than overcounting.
  • When a team starts quoting calculator numbers in budget meetings instead of invoice numbers. This is the failure mode: the abstraction replaces the receipt.
  • When the tool has no track record of updating within 48 hours of a provider price change. Stale pricing is worse than no pricing because it carries false confidence.
  • When you are already running production workloads. At that point, your own billing data is the only calculator that matters. Switch from estimation to measurement.

So what

The gap between a calculator estimate and your actual invoice is itself a diagnostic tool. If the gap is small, you understand your workload. If the gap is large, you don't — and that misunderstanding is more expensive than any calculator error.

Most teams skip this comparison. They run the calculator, get a number, and move on. The surprise bill later is not the calculator's fault. It is the cost of never closing the loop between estimate and measurement.

Never let a calculator number survive past your first real invoice. The first invoice is the moment to recalibrate. If you don't, every subsequent estimate drifts further from reality while feeling increasingly precise.