AI cost dashboards often stop at provider, model, tokens, and day. That is enough to reconcile an invoice. It is not enough to govern an agent fleet.
An agent can retry, call several models, retrieve documents, invoke paid tools, run code, store artifacts, and delegate to other agents. The model call may be only one part of the cost. A cheaper model can still produce an expensive workflow if it loops or fails often.
Cost visibility has to follow the run from request to outcome.
Attribute cost to the work
For each run, capture:
- —agent and workflow version;
- —owning team and cost center;
- —initiating user or system;
- —environment and customer or tenant boundary;
- —model input, output, cache, and reasoning usage where available;
- —retrieval, tool, compute, storage, and connector costs;
- —retries, delegated runs, and fallback models;
- —latency, completion status, and business outcome.
Roll those records up by agent, team, use case, provider, model, and environment. Keep the run identifier so finance can drill from a monthly total into the behavior that created it.
Measure cost per useful outcome
Tokens are a resource unit, not a business result. Pair spend with an outcome the workflow owner recognizes: reviewed case, resolved request, processed document, completed control check, or accepted recommendation.
Track failure and rework separately. A run that consumes tokens but times out is not equivalent to one that finishes. A run that completes but is rejected by a human reviewer may be more expensive than its provider cost suggests.
Do not invent a universal ROI formula. Define the outcome and value model with the business owner, then keep the technical cost record honest.
Set budgets at several levels
One monthly account limit is too blunt. Use layered controls:
- —per-run token or currency ceiling;
- —maximum steps, retries, and delegated agents;
- —per-agent daily or weekly budget;
- —team or cost-center budget;
- —model and tool allowlists by risk tier;
- —anomaly alerts for sudden changes in volume or unit cost.
Decide what happens at the boundary. The runtime may stop, degrade to a cheaper path, require approval, queue the work, or notify an owner. Silent overages and silent truncation are both poor defaults.
Keep cost attached to policy
Cost is a governance signal. A policy decision can consider the agent, environment, requester, task type, current budget, and expected consequence before allowing an expensive path.
Record that decision with the run. When a reviewer asks why an agent used a premium model or exceeded its normal limit, the answer should be a visible policy or approved exception, not a guess.
Watch for cost drift after changes
Prompt updates, model migrations, new tools, larger context windows, and altered retry logic can change unit economics without changing user-facing functionality.
Treat cost as a release metric. Compare representative runs before and after a material change, then monitor the production distribution. The goal is not always to minimize spend; it is to know what additional cost bought and whether the owner intended it.
Provider billing tells you how much AI cost. Run-level evidence tells you what the organization paid for.