An enterprise agent has eighty cents left in its task budget. Twenty-four workers read that balance, each decide that a ten-cent request fits, and dispatch together. Every worker followed its local rule. The shared budget still failed.
The control has to happen before dispatch: reserve the maximum permitted cost of each attempt against a shared ledger, carry that reservation through uncertain outcomes, and settle it only when reliable usage evidence arrives. All child agents and retries must draw from the same accountable budget hierarchy.
That design can bound admitted variable spending, but only when requests have defensible cost ceilings and every spending path passes through the control. A dashboard, an average cost estimate, or a warning in the system prompt cannot provide that guarantee.
Decide what the budget actually promises
Start with a narrow contract: this workflow may consume at most a specified amount of variable inference and metered tool charges. Name the currency, billing period, included services, rate-card version, and treatment of taxes, discounts and credits. Keep fixed platform costs separate unless the allocation model explicitly includes them.
There are three different controls here. Rate limits protect service capacity. A task allowance limits how far one piece of work can expand. Billing budgets monitor or restrict costs at a provider-defined scope. Use all three, but do not assume they share a counter or an enforcement instant.
AWS Budgets documents a delay between incurring charges and receiving notifications; further costs can accrue before an alert arrives. That makes alerts useful for oversight and reconciliation, not sufficient for deciding whether the next parallel call is affordable.
Provider enforcement also deserves a precise reading. Google Cloud spend caps can pause new usage for eligible services, but their documentation says enforcement is not instantaneous, in-flight requests finish and accrue applicable charges, and persistent resources can continue costing money. A provider cap is a valuable backstop. Its boundary is different from a business case’s remaining allowance.
For an agent platform, define budgets at three levels: tenant or department, workflow run, and individual attempt. A child agent inherits the run’s identity and ceiling; spawning a child does not mint a new allowance. The model may propose a larger task, but a trusted workflow service decides whether that work receives more budget.
Put the ledger beside the credentialed dispatcher
Use a path such as:
1 | workflow / child agent |
The dispatcher owns provider credentials. Workers cannot bypass it with a direct SDK call or their own API key. The tenant and workflow IDs come from authenticated execution context, not from model-generated arguments. Batch jobs, embeddings, reranking, paid searches and provider-managed tools need the same scrutiny as the main chat endpoint.
For each applicable budget account, keep:
1 | B = authorized ceiling |
This is an accounting invariant for the proposed architecture, not a promise made by an LLM API. Store amounts in integer minor accounting units, such as microdollars, with an explicit currency. Round reservations upward; avoid floating-point comparisons at the admission boundary.
The check and increment must be one atomic operation. Reading a balance and then updating it in separate transactions recreates the opening race. A reservation spanning department and run budgets must succeed at both levels or neither. Use a transaction boundary that supports this, or allocate disjoint spending envelopes to runs in advance.
If a parent has already reserved an envelope for a child, child spending consumes that envelope. It is not an additional charge against the parent. Keep the hierarchy explicit so an implementation neither spends the same allowance twice nor counts the same reservation twice.
Regional gateways need particular care. Azure API Management’s LLM token-limit policy tracks usage independently at each gateway and does not aggregate it across the entire instance. Its documentation also notes that concurrent requests can temporarily exceed the configured token limit. Do not interpret those token counters as a globally serialized monetary ledger.
A shared ledger can add latency and become an availability dependency. One alternative is to allocate non-overlapping amounts to regions, subtracting those allocations from the central available balance first. A disconnected region can spend only its existing allocation. Rebalancing requires transfer and fencing rules that prevent both regions from using the same funds. Copying a global balance into two caches is not an allocation scheme.
Bound the request before pricing it
The reservation must cover what the dispatcher will actually send. Bind it to the route, immutable request digest, output limit, enabled features and pricing version. A changed model, larger context or extra tool requires a new calculation before dispatch.
For a simple route, the calculation might be:
1 | reservation = maximum billable input units × input rate |
The difficult word is maximum. A forecast based on average completion length is useful for capacity planning, but it does not reserve the cost of a long completion. Enforce the route’s output cap and verify which generated units, including reasoning where applicable, that cap actually constrains.
Count the assembled request after adding system instructions, retrieved context, schemas and tool results. Anthropic’s token-counting documentation explicitly describes the result as an estimate that can differ from actual input usage. A small empirical margin reduces ordinary estimation error; it does not convert an estimate into a proven upper bound.
For a strict admission class, use a documented bound on accepted billable units, or a deliberately conservative route maximum that covers the permitted request. If the provider’s accounting cannot be bounded for a feature, exclude that feature from this class. A separate class may accept bounded business risk with monitoring, but its owner must understand that the ceiling is a target rather than a guarantee.
Caching complicates the rate card. Reserve the most expensive permitted outcome, including any cache-write surcharge, rather than assuming every input gets a cache hit. Apply discounts after receiving evidence. Similarly, do not price an entire autonomous server-side tool session as one ordinary completion if it can incur additional calls or fees outside the chosen cap.
Every retry needs its own exposure calculation. The same applies to a fallback model: a previous attempt may still be consuming resources while a new route starts. The eligibility checks in model failover architecture still apply; cheaper remaining capacity does not make an otherwise unsuitable route acceptable.
Work the parallel case with real accounting states
Consider an illustrative workflow with a $10.00 allowance. It has settled $7.60, and unresolved or running calls hold $1.60. Available capacity is $0.80.
Suppose the chosen request profile permits at most 12,000 billable input tokens, 2,000 billable output tokens, and $0.012 of additional metered work. At hypothetical rates of $4 and $20 per million input and output tokens:
1 | input: 12,000 × $4 / 1,000,000 = $0.048 |
These are design numbers, not vendor prices or measured results. The additional work is assumed to have an enforced quantity limit; an uncapped tool would invalidate the example.
Eight new attempts can reserve ten cents each. The ninth waits or receives a budget-exhausted result. If all twenty-four workers merely checked the same eighty cents, their new exposure could reach $2.40, bringing settled charges plus total exposure to $11.60.
When one ten-cent attempt finishes with a verified charge of $0.055, settlement moves $0.055 into S, removes $0.100 from R, and returns $0.045 to available capacity. Replaying that same receipt must have no further effect. If actual usage exceeds the reservation, record the real charge, stop new admission for the affected scope, and investigate the broken bound. Hiding the excess preserves a neat counter while destroying the control.
Reserve per attempt when the workflow can checkpoint between steps. Reserve a larger envelope before starting when the process requires several steps to finish usefully. Neither approach guarantees task success. A capped workflow must be allowed to stop with incomplete work.
Treat missing receipts as outstanding exposure
Timeout handling is where an apparently strict budget often becomes optimistic. A client disconnect is evidence that the client stopped waiting. It is not sufficient evidence that remote work stopped or that no charge will arrive.
Use a durable reservation state machine with a deliberately uncomfortable state for ambiguity:
| State | Budget treatment | Recovery condition |
|---|---|---|
| Reserved, unsent | Full amount held | Release only after dispatch is fenced out |
| Dispatch started | Full amount held | Await outcome; do not blindly resend |
| Usage uncertain | Full amount held | Reconcile evidence or conservatively debit the ceiling |
| Settled | Actual charge counted | Duplicate receipts have no effect |
| Cancelled before send | No exposure | Evidence that no dispatcher can still send |
Write the dispatch transition durably before invoking the provider. If the worker crashes around the network send, recovery may not know whether the provider accepted the request. Keep the hold and investigate. A reservation identifier deduplicates bookkeeping; it does not make the provider call idempotent.
Disable hidden SDK and proxy retries unless each possible billable transmission is covered. A new transmission normally receives a new reservation. Reuse an attempt only when the downstream API’s documented idempotency contract makes that safe for the operation. The distinction between an attempt ID and a business operation ID follows the same reasoning as enterprise agent tool contracts.
Do not refund a dispatched reservation merely because its lease or TTL expires. A lease controls worker ownership, not provider billing. For an unsent reservation, reclamation requires fencing the old dispatcher so it cannot wake later and spend a released allowance. For uncertain usage, keep the hold or convert its full amount to a conservative charge; refund only on evidence. If the actual exposure was never bounded, neither treatment repairs that missing guarantee.
Budget periods need stable identities too. An attempt admitted just before midnight retains its original account-period reference when its receipt arrives after midnight. If the contract is instead based on the provider’s billing-period attribution, retain and reconcile overlapping exposure across the boundary. Do not erase unresolved calls during a counter reset.
Make exhaustion a normal workflow outcome
When funds run out, checkpoint the business case and stop creating paid work. Return a structured reason such as budget_exhausted, the accountable run, and whether reconciliation or an authorized increase could allow progress. Avoid asking the same model to summarize why it cannot afford another model call.
If producing a final response is necessary, earmark a small completion allowance before exploratory work begins. That allowance remains part of the total budget and cannot be spent twice. Durable checkpointing and cancellation should work without another inference request. Completed tool actions remain completed even when later reasoning becomes unaffordable.
The workflow owner decides which work may stop, which may wait, and who can raise the ceiling. The platform team owns enforcement and reconciliation. Finance or FinOps owns the rate-card and billing mapping with the platform team. This makes the cost responsibility in the CIO operating model concrete enough to operate during an incident.
Keep budget events in the agent audit trail: reservation ID, parent run, account period, route, request digest, pricing version, held amount, provider request ID, settlement evidence and any override. Avoid logging raw prompts just to explain a charge.
Monitor unresolved exposure and its age alongside settled spend. A system with low settled costs and growing unknown holds may be losing receipts, not becoming more efficient. Reconcile ledger totals against provider usage and invoices to detect bypass routes, missing billable categories and price changes. Report cost per accepted business outcome as well as cost per attempt, so cheap failed work does not look like progress.
Test the failures that can spend the same allowance twice
Use a provider stub with controlled usage receipts for repeatable accounting tests, then qualify the real route’s token caps and billing behavior separately. A correct ledger cannot compensate for an incorrect cost bound.
| Fault to inject | Required evidence |
|---|---|
| Twenty-four simultaneous admissions with $0.80 free | At most eight $0.10 reservations succeed |
| A child starts another child | Both consume the original run’s allowance |
| Crash immediately around dispatch | Exposure stays held; recovery does not silently resend |
| Timeout followed by fallback | Both possible charges remain covered |
| Duplicate or late usage receipt | Settlement applies once to the original account period |
| Price, model or request changes | Old reservation cannot authorize the changed request |
| Region loses the central ledger | New spending stops or uses only its disjoint allocation |
| A stale worker wakes after reclamation | Its dispatch authority is rejected |
Also deliberately remove a cache hit and enable a costly feature in a test environment. The first should fit the reserved worst case; the second should be rejected unless the request is repriced and admitted. Attempt a direct provider call from a worker to prove that the gateway is an enforced boundary.
The release decision should rest on evidence that concurrent callers, crashed workers and late receipts preserve the accounting invariant. Once those paths are controlled, an agent can explore within a real spending envelope—and stop with an honest partial result when that envelope is exhausted.