An AI invoice shows what was billed, but it may not show why each call was made. To decide whether architecture or provider pricing is the better lever, connect billed usage to the task and execution path that produced it.
Architecture affects how often a system calls a model, what it sends, how much it asks the model to generate, and how often work is retried or repeated. Provider rates determine the price of each billable unit. A useful audit measures both; it does not assume one is always the larger opportunity.
Map work before comparing rates
Start with a representative period and group requests by task class. For each class, trace the workflow from the initial task through model calls, tools, retries, fallbacks, and evaluation. Record the model or service, input and output usage, attempt number, task purpose, and outcome-quality check.
OpenTelemetry's GenAI semantic conventions define telemetry for input and output token usage, including cache-related input tokens. These fields describe usage; they are not a provider billing ledger. Reconcile instrumentation with the provider's bill for the same period because billable units and rates depend on the provider, model, contract, and service.
For token-priced calls, estimate charges from the provider's mutually exclusive billing categories. Treat a total input-usage field as including any cached subset unless the provider's billing export defines separate, non-overlapping categories; do not add cached input to that total again. Partition usage into uncached input, cache reads, cache writes, or the provider's actual categories only when the provider bills them separately, then apply the rates in effect for that period. Use actual billing dimensions for services charged by another unit, and reconcile telemetry to the provider bill.
Separate three causes of spend
Architecture and workload. Count calls per task, input and output usage, repeated requests, retries, fallbacks, and evaluation calls. A retry is additional work, but not automatically waste: it may recover a failed task. Keep its reason and outcome so the audit can distinguish recovery from a recurring first-pass defect. For context size, compare the material sent with the information the task needs; do not infer quality from token count alone.
Provider rate. Apply the rate card or contract that was active during the measured period to the actual billed units. Then hold that usage constant while modeling an alternative rate. This isolates a pricing change from changes to the workflow. Do not compare headline rates across models without checking input/output categories, cached units, and other billing terms.
Other and shared costs. Account separately for retrieval, storage, orchestration infrastructure, and evaluation systems that are outside the model charge. Attribute a cost directly only when the records support it. If infrastructure or evaluation is shared, show the allocation rule and any unallocated amount instead of presenting an estimate as exact per-task or per-customer cost.
Caching needs the same precision. An application result-cache hit may skip a model request; provider prompt caching may change the billed rate for part of a request while the call still occurs. Measure the actual hit category, billable usage, freshness behavior, and output quality. A cache does not guarantee a saving or a correct result.
Test architecture changes against the same work
Use a task set that represents the workload and quality requirements the system must keep. Compare the current workflow with one proposed change while holding the task mix, evaluation checks, and applicable rates constant. Measure successful completion, output quality, billable usage, retries, and latency. Then model a rate change separately against the same measured usage.
For example, if a workflow repeats the same retrieval and model call after a tool timeout, compare a retry-policy change with the current path on the same cases. Count any additional attempts and check whether the response still meets the required quality. If a model-routing change is proposed, test each task class before assigning work to a cheaper model. A lower bill without the same output checks does not show that the system did equivalent work.
Raw totals from periods with different task volume or task mix are not directly comparable. Normalize the comparison to the same task classes and state what remains outside the measurement. If the request path and usage are already understood, price comparisons can be decisive. If attribution is missing, fix the measurement before ranking either lever.
For context-selection decisions, see Context Engineering for Production AI Agents. For event-level records around consequential agent actions, see What To Log Before An AI Agent Gets Write Access.
The audit's result is a traceable comparison: what the system did, what it was billed for, which quality boundary must hold, and which proposed change affects usage, rate, or both. Architecture comes first in the investigation because it explains the work being priced; pricing remains a separate measured lever.