Architecting Usage-Based Billing for AI SaaS: Tracking Tokens Without Sinking Query Performance

The Margin Trap of Flat-Rate AI Subscriptions

When software teams build AI features into their SaaS products, the first instinct is usually to charge a flat $29 or $49 per user per month. It keeps sales simple and marketing clean. But that simplicity disappears the moment three enterprise accounts start running complex recursive LLM agents across 2,000 PDF documents a day. Because API costs scale linearly with prompt and completion tokens, flat pricing guarantees that your heaviest power users will swallow your gross margins whole.

The obvious technical fix is usage-based or hybrid billing: give tenants a base tier of included tokens, and meter everything above that limit. But building an accurate meter engine for high-concurrency LLM operations introduces a severe database architecture problem. If your backend attempts to update user balances or log token counts in PostgreSQL synchronously during an active Server-Sent Events (SSE) streaming response, you will quickly hit catastrophic connection pool exhaustion and write lock contention.

The Multi-Tier Async Token Metering Architecture

To meter usage reliably without adding 150ms of overhead to every user request, you must decouple token accounting into three separate tiers: real-time in-memory accumulation, durable local persistence, and external billing synchronization.

Tier 1: In-Memory Ingestion via Redis Atomic Operations

When an LLM endpoint returns a final response chunk containing token usage metadata (or when an internal tokenizer like tiktoken finishes counting), your application code should never touch a relational database. Instead, write immediately to a high-throughput Redis instance using atomic commands.

We use a combination of Redis Hashes and Sorted Sets. For instantaneous quota enforcement (checking if a tenant has exceeded their monthly threshold before initiating a call), we use HINCRBY on a daily tenant key. For raw event logging, we push a lightweight JSON payload to a Redis Stream key using XADD.

Because Redis handles these atomic commands in microsecond timescales, logging token consumption adds less than 2ms to the HTTP request lifecycle. The application worker returns its response stream to the user without waiting for disk writes or external billing network calls.

Tier 2: Durable Persistence via Postgres Write-Behind Workers

While Redis is incredibly fast, it is not your system of record. If a Redis node fails before flushing data, you lose billing data—and in SaaS, losing billing data is losing real money. To secure permanence, a dedicated queue consumer process reads from the Redis Stream in batches using XREADGROUP every 5 to 10 seconds.

This worker process aggregates hundreds of small token events into a single bulk PostgreSQL insert statement using INSERT INTO usage_events ... ON CONFLICT DO NOTHING. By inserting 500 meter events in a single database transaction rather than 500 individual queries, you reduce write IOPS on your primary database cluster by over 90%.

This architecture is identical to the durable event logging patterns we implemented across our engineering projects when building local-first data processing engines. Keeping heavy write activity isolated in dedicated background threads keeps primary web application queries fast and deterministic.

Tier 3: Asynchronous Metering Sync to Stripe

Once raw token events are safely persisted in PostgreSQL, the final step is reporting those figures to your payment gateway. Trying to call external APIs synchronously within the web request path is an anti-pattern; if the external endpoint experiences an outage or elevated latency, your application code hangs.

We schedule an asynchronous job using BullMQ or Celery to run every hour. This job queries the local PostgreSQL event log for un-synced consumption records, calculates the aggregate units (e.g., thousands of input tokens vs output tokens), and transmits them to external APIs like Stripe's official meter events documentation.

Handling Edge Cases: Idempotency, Outages, and Race Conditions

Building a billing pipeline means accounting for distributed system failures. Three specific edge cases must be explicitly engineered into your metering pipeline:

• Duplicate Ingestion: Network timeouts between your application server and LLM providers can lead to request retries. Always tag every LLM call with a unique, deterministically generated request_id (UUIDv4) at the web gateway level. Store this key in your Redis event ledger to prevent counting the exact same generation twice.
• Stripe API Outages: Network partitions or rate limits (HTTP 429) from payment APIs are inevitable. Your background sync task must be completely stateless and retry-safe. Store a synced_at timestamp and a stripe_event_id on your PostgreSQL ledger rows. If a batch sync fails, the worker simply picks up the un-synced rows on the next execution interval.
• Hard vs. Soft Quota Enforcement: Hard blocking a enterprise tenant mid-conversation because they exceeded their monthly limit by 12 tokens creates terrible user experiences. Implement a double-threshold system: trigger an email alert when a workspace hits 80% and 100% of their base quota, but allow a 10% overflow margin while asynchronous billing upgrade prompts are delivered in-app.

Keep Billing Code Out of the Critical Hot Path

Pricing models for AI SaaS are evolving rapidly. Today you might charge per token; tomorrow you might charge per agent execution or per megabyte of context processed. By building a clean three-tiered decoupling—Redis for speed, Postgres for durability, and queue-driven workers for external billing sync—you make your pricing layer flexible enough to change on a dime without risking system performance or data integrity.

Synchronously inserting a record into PostgreSQL on every streaming LLM token chunk is a fast track to connection pool exhaustion and catastrophic database lock contention.