On August 1, I tested a runaway prompt loop against my old Google Cloud Billing alert setup. In that setup, a Pub/Sub Google Cloud's message service. One program publishes a message, and every program subscribed to it receives it. budget alert starts a Cloud Function A small piece of my code that Google Cloud runs when an event arrives, such as a message., and the function disables billing on the project. Billing came off only after my script burned $412.00 above my $50.00 budget. In September 2026, Firebase and Google Cloud shipped native Spend cap A monthly dollar limit on one service in one project. When spend reaches it, Cloud Billing pauses that service.. Jeff Huleatt called them a big deal, because they actually pause a supported service when its cap is reached. Even so, a billing spend cap alone will not save my application from a runaway prompt loop.
A native spend cap guards one eligible service in one project. It knows nothing about my users or their sessions. My agents run many turns on their own. Say I deploy caps in front of them with no Rate limiter Code in my app that counts each user's requests and refuses new ones once that user passes a limit. in my app. Then the cutoff stops the whole service for everyone, and it can leave my data half written. So I protect my production workloads with a real-time, three-layer tokenomics defense. It stops abusive Token A small piece of text, often part of a word. Models read, write and bill text by the token. use inside my app, before the spend ever reaches cloud billing.
The asynchronous billing metering lag and state traps
Pipeline comparison: native spend caps vs. legacy alerts
Cloud Billing Metering How Cloud Billing measures what each service used and turns it into cost. It runs behind my traffic. my spend in its own pipeline, which runs behind my traffic. That pipeline is asynchronous: it reports a cost some time after the request that caused it. So every billing cutoff trails the spend it reacts to. The official Google Cloud spend caps documentation and the budget notification docs describe three ways a cutoff can work. Here they are, in Google's own words:
| Billing Cutoff Pipeline and Scope | Documented Lag | What Happens at the Cutoff |
|---|---|---|
| Pauses one service. Firebase spend cap (AI Logic, App Hosting, Cloud Functions for Firebase, Extensions; underneath: Gemini API, Cloud Run, Cloud Run functions) Scope: one service in one project, for every caller of it | Minutes late. Not "hard caps": enforcement "can be delayed by several minutes"; the Gemini API cites about 10 | Every user of that service is cut off. New requests fail and every user's client gets errors, in-flight requests finish, and the overage is billed |
| Sends an email. Alerts-only budget (the only option for Firestore) Scope: the projects and services I pick | Can be hours late. The first email "may take several hours" | Nothing stops. An email; nothing pauses |
| Turns off billing. Legacy Pub/Sub budget alert (wired to Google's disable-billing sample) Scope: detaches billing from the whole project | Can be hours late. The first notification "may take several hours", then several a day, delivered at least once | The whole project stops. Every resource in the project shuts down; Google warns "Resources might be irretrievably deleted" and that the sample "doesn't guarantee that you won't spend more than your budget" |
Even with a Firebase spend cap on every eligible service, my three-layer tokenomics defense remains mandatory for three physical reasons:
- Minutes of lag still overshoot: Everything my loop spends before enforcement lands is billed. So the overshoot is my loop's burn rate times the lag. The Gemini API page also warns that agent sessions "may incur overages beyond your project spend cap."
- Whole-service outage: When a spend cap trips on App Hosting or Cloud Run, the service stops taking new requests. Every customer's calls fail until I lift the cap or the month ends. Layer 2 is a per-user Token bucket A per user allowance that refills at a steady rate. Each request spends from it, and an empty bucket refuses. (mine is a Firestore transaction). Without it, a single runaway user session or stuck agent loop takes down my entire production app for every customer.
- Mid-chain state orphaning: In-flight requests finish, but the next call in a multi-step tool chain fails. The earlier steps stay committed, with no rollback.
To watch the lag and the whole-service outage together, I built the spend-cap fuse simulator below. It runs an overnight agent against my simulated $50.00 spend cap. I can set the reporting lag from 5 to 20 minutes, around the documented 10. Its chart zooms in on the minutes around the $50.00 line. There the lag shows up as a gap between my spend and the billing meter. Then I compare the cap alone with a per-user application circuit breaker:
Run an agent loop all night against a $50 cap and read the invoice line that shows how many dollars landed after the cap fired.
The multi-turn agent state trap
When a spend cap trips, Cloud Billing pauses that one service in that one project. Requests already in flight run to completion. Every new request fails, so the client app starts throwing errors, whether it calls Firebase AI Logic or an App Hosting backend. My agent loop breaks at its next call. If I leave the cap alone, the service stays paused until the cap resets on the 1st. If I lift it, the service can take up to an hour to resume. Then it runs with no limit for the rest of the month, unless I raise the cap.
Here is what that does to an agent in the middle of a task. Say my agent is in Step 3 of a 4-step tool execution chain. It wrote a state update to Cloud Firestore and was about to trigger an external Webhook A web address that another service calls when something happens, so code there can react.. The pause fails Step 4. Multi-tool LLM Large language model. A model trained on huge amounts of text to predict the next token. loops lack native ACID Atomic, consistent, isolated, durable. A database promise that each transaction lands whole or not at all. boundaries. So a blunt infrastructure pause, without an application-level circuit breaker, creates orphaned, inconsistent database records in my production environment.
Get the next essay by email
One email per new essay: a production failure, taken apart, and the code that fixed it. Reply to any of them to unsubscribe.
The three-layer tokenomics defense architecture
To protect my production AI systems, I stack three distinct layers of defense. The first sits at ingress, where requests enter. The second runs in my application code. The last one is in billing infrastructure:
- Cryptographic Device and Client Attestation. I validate client Attestation Proof, signed by the device or platform, that a request comes from my real app and not from a script. tokens at the edge. That blocks unauthorized automated bots and malicious scripts before expensive model inference runs.
- User-Level Quotas and Circuit Breakers. I charge each request's estimated tokens to a per-user token bucket in a Firestore transaction. Afterwards I settle the real input and output count with an Atomic increment A database update that adds a number in one step, so two updates at the same moment never lose one.. When a bucket is empty, I return a structured application-level rate limit instead of an abrupt infrastructure crash.
- Service Spend Caps. I set one Firebase spend cap per eligible service, plus an alerts-only budget for Firestore, which caps don't cover. Each cap pauses only its own service, minutes late. It matters only if the token buckets and edge checks upstream are breached.
Application-layer rate limiting in Cloud Firestore
I do not wait for a spend cap to pause a whole service. Instead, I keep a per-user token bucket at the application layer in Cloud Firestore. It refills continuously. It answers an empty bucket, or a transaction that loses to contention, with allowed: false. It also rejects a malformed estimate or cap:
// Per-user single-rate token bucket (RFC 2697's committed bucket only): refills continuously, refuses when empty
import { Firestore, FieldValue, Timestamp } from "@google-cloud/firestore";
const db = new Firestore();
const DAY_MS = 86_400_000;
export async function checkAndDeductTokens(
uid: string, // from the verified ID token, never from the request body
estimatedTokens: number, // an upper bound: countTokens on the prompt + maxOutputTokens
maxDailyTokens: number
): Promise<{ allowed: boolean; remaining: number }> {
// A NaN cap (an unset env var) would make every comparison false and admit everything
for (const [name, n] of Object.entries({ estimatedTokens, maxDailyTokens })) {
if (!Number.isInteger(n) || n <= 0) throw new RangeError(`${name} must be a positive integer`);
}
const bucketRef = db.collection("token_buckets").doc(uid);
return db.runTransaction(async (transaction) => {
const bucket = (await transaction.get(bucketRef)).data();
const now = Date.now();
// Refill at maxDailyTokens per day, never above one day's worth (so any 24 hours admits up to 2x)
const tokens = bucket
? Math.min(maxDailyTokens, bucket.tokens + ((now - bucket.updatedAt.toMillis()) * maxDailyTokens) / DAY_MS)
: maxDailyTokens;
if (tokens < estimatedTokens) {
return { allowed: false, remaining: Math.floor(tokens) }; // the caller answers 429, not a crash
}
transaction.set(bucketRef, { tokens: tokens - estimatedTokens, updatedAt: Timestamp.fromMillis(now) });
return { allowed: true, remaining: Math.floor(tokens - estimatedTokens) };
}).catch((err: { code?: number }) => {
if (err.code === 10) return { allowed: false, remaining: 0 }; // ABORTED after retries: contention, answer 429 too
throw err;
});
}
// After the model call, charge real usage: usageMetadata.totalTokenCount. A timed-out call may still
// bill, so settle it at estimatedTokens; pass 0 only when the request provably never reached the model
export async function settleTokens(uid: string, estimatedTokens: number, actualTokens: number): Promise<void> {
await db.collection("token_buckets").doc(uid).update({ tokens: FieldValue.increment(estimatedTokens - actualTokens) });
}The bucket is only as tight as its inputs. The estimate has to be an upper bound: countTokens on the prompt plus maxOutputTokens. That is because concurrent calls are admitted against their estimates before any of them settles. A call that times out on my side may still be billed by the provider. So I refund an estimate only when the request provably never reached the model. The cap is a refill rate, not a calendar window. A full bucket plus a day of refill lets one user spend up to twice maxDailyTokens in any 24 hours.
Industry validation and benchmarks
- Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms (Aug 2026): Confirms that unthrottled burst token generation creates non-linear cost spikes that asynchronous cloud telemetry cannot bound without synchronous ingress rate limiting.
- PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response (Aug 2026): Confirms that enforcing synchronous prefill/decode admission control at the serving gateway prevents resource and budget exhaustion during traffic surges.
- IETF RFC 2697: A Single Rate Three Color Marker (Token Bucket Algorithms)
- Google Cloud Spend Caps and Billing Quota Architecture
- Firebase Service-Level Spend Caps
- Firebase App Check Device and Client Attestation
- Cloud Firestore Transactions and Atomic Increments
- Jeff Huleatt: Cloud Spend Caps for Firebase Architecture