Token cost breakdown per feature and customer
The idea
Every founder who bolted an AI feature onto their SaaS eventually has the same bad afternoon: the OpenAI invoice doubled, and the bill is one opaque number with no way to tell which feature or which customer did it. The nightmare case is specific, one whale customer hammering an AI feature hard enough to make their account margin-negative, invisible until the month closes.
Langfuse and the observability platforms technically contain this data, but they are built around traces, evals and prompt engineering, and answering who is costing me money means building your own queries. The wedge is doing one thing at the founder’s altitude: a pure cost ledger behind a base-URL swap and two headers, with a dashboard whose first table is customers sorted by what they cost you.
Who pays
The buyer is the founder or first engineer of a SaaS with AI features and real customers, converting at the moment the invoice first surprises them. At 29 to 99 dollars a month it needs to save one mispriced customer to pay for itself for years. Distribution: SEO on openai bill by customer and per-model pricing pages, build-in-public X where invoice screenshots already go viral, and AI-founder communities.
MVP scope
- Drop-in proxy: base-URL swap, provider-key passthrough, feature/customer/deploy headers, streaming-safe
- ClickHouse event store of usage metadata only, no prompt bodies
- Dashboard: spend headline, cost-per-customer, cost-per-feature, cost-per-deploy
- Margin alerts against user-entered per-customer revenue
- Maintained model-price table covering OpenAI and Anthropic
- Skip for v1: SDK wrappers per language, tracing/evals/prompt storage, self-hosted proxy, Azure/Bedrock/Gemini providers, seat-based team permissions
Build TokenLedger, a cost-attribution proxy for LLM API spend. Stack: Next.js for dashboard and proxy edge routes, ClickHouse for the event store, Stripe for billing. Core flow: integration is a base-URL swap plus two headers. The customer points their OpenAI/Anthropic SDK at proxy.tokenledger.app with their TokenLedger key, keeps their provider key in an Authorization passthrough, and adds X-TL-Feature and X-TL-Customer headers to each call site. The proxy forwards requests unchanged (streaming included: pass chunks through untouched and read usage from the final SSE event), then writes one row per call to ClickHouse: timestamp, provider, model, feature, customer_id, optional X-TL-Deploy sha, prompt/completion/cached tokens, computed cost from a maintained model-price table, latency, status. Never store prompt or completion bodies; this is a ledger, not a trace store. Dashboard, designed to be read in 30 seconds: today/this-month spend headline with delta vs last period, cost-per-customer table sorted descending (the whale finder), cost-per-feature breakdown, model mix, and cost-per-deploy so a regression that doubled token use shows up against its sha. Margin alerts: user enters what each customer pays per month; email/Slack alert fires when a customer's LLM cost crosses a set percentage of their revenue. Pages: landing with a two-line integration snippet above the fold, dashboard, alert config, model-price table (public, good SEO), billing. Pricing: free to 50k calls/month; 29 GBP/month to 1M; 99 GBP/month beyond plus 90-day retention.
// More Dev tool ideas
-
Deprecation countdowns for the APIs you use
Stripe and Shopify deprecations hide in changelogs until things break; this scans your code for affected calls and gives you a countdown.
Dev tool 1 month -
App store reviews filed as GitHub issues
Bug reports buried in app store reviews never reach the tracker; this clusters new reviews and files deduped GitHub issues nightly.
Dev tool 1 week -
Nightly restore drills for database backups
A backup you've never restored is a guess; this restores your Postgres dump to a scratch instance nightly and proves the data is intact.
Dev tool 1 week -
Hosted monitoring and alerts for BullMQ queues
A stuck BullMQ queue can silently eat jobs for days; hosted dashboards, dead-letter alerts and retry controls without running Grafana.
Dev tool 1 week