AI Agent Observability: What to Measure in Production

An AI agent is not production-ready because its final answer looks plausible. AI agent observability means following a task from the user request through model calls, retrieval, tool execution, approvals and the final outcome—and explaining latency, cost, failure and risk at each step. The practical starting point is one correlated trace per task, with structured spans for every model and tool operation.

That view changes the question from “Did the model respond?” to “What happened, under which version, with which permissions, at what cost, and with what result?”

Why AI agent observability is different

Traditional monitoring follows deterministic services and known request paths. An agent adds a variable decision loop: it can select a tool, retry, ask for approval, retrieve context or stop early. The same request may therefore produce different paths and costs.

Observability should make that path inspectable without turning sensitive prompts, retrieved documents or customer data into a second security problem. OpenTelemetry’s GenAI work is useful because it treats model and agent operations as telemetry that can be correlated with the rest of the application.

Observability is not evaluation. Evaluation asks whether an answer or trajectory was good. Observability records what the system did so teams can diagnose, compare and govern it. Production needs both.

The five signals to measure

1. End-to-end traces

Create a trace for the user task or business job, then nest the operations that made it happen. At minimum, distinguish:

agent or workflow orchestration;
model invocation, including provider, model version and response status;
retrieval and reranking;
external tool calls, arguments and return status;
policy checks, human approvals and retries;
final response and business outcome.

A trace should show whether the failure came from the model, context selection, a tool, a permission boundary or the application around the agent. Use a correlation ID separate from a person’s stable identity, and propagate it across queues and services.

OpenTelemetry’s GenAI observability overview describes this hierarchy. Its GenAI semantic-conventions repository is the working reference for teams standardising attribute names across providers. Conventions evolve, so pin the version you adopt and record the schema version in telemetry.

2. Latency, tokens and cost per task

Model duration is only one part of agent latency. Track time spent waiting for a tool, in a queue, in retrieval and in human approval. Report p50 and p95 latency for the complete task, not only the fastest model span.

For each model operation, capture input tokens, output tokens, cached tokens when available, retry count, model identifier and provider. Aggregate these into cost per completed task and cost per successful business outcome. A cheap run that ends in manual escalation is not necessarily cheaper than a longer run that completes correctly.

View	Minimum measures	Decision it supports
Reliability	task success, error, timeout and escalation rates	Is the workflow dependable?
Performance	p50/p95 task latency, tool latency and queue time	Where is the user waiting?
Economics	tokens, retries and cost per task/outcome	Is usage sustainable?
Control	denied actions, approval rate and policy events	Is autonomy bounded?

3. Tool and permission behaviour

Tool use is where an agent can move from generating text into changing systems. Record which tool was selected, whether the call was allowed, its duration, result class and permission context. Prefer typed, redacted summaries of arguments over raw payloads.

Measure tool-level failures separately from model failures. A reported “model error” may actually be an expired API token, rate limit or downstream schema mismatch. The distinction matters for remediation and incident review.

The OWASP AI Agent Security Cheat Sheet recommends least privilege, explicit validation and monitoring of agent actions. Telemetry cannot replace those controls, but it shows whether controls are exercised and where the agent repeatedly reaches a boundary.

4. Quality and outcome signals

A complete trace is not evidence that the result was useful. Join technical telemetry with an outcome signal appropriate to the workflow:

a support case resolved without re-open;
a document classified and accepted by a reviewer;
a code change that passes tests and review;
a research answer that cites approved sources;
a human correction, rejection or escalation.

Keep these signals explicit. “The user did not complain” is a weak proxy and can hide silent failure. For high-impact workflows, sample traces for human review and compare production outcomes with an evaluation set. Store the review result as a separate event so quality can be analysed without rewriting the original trace.

5. Safety, privacy and auditability

Prompts and tool outputs may contain credentials, personal data, confidential documents or malicious instructions. Default to metadata and redacted excerpts; capture full content only when there is a documented reason, access control, retention policy and deletion path.

Record the versions that affect behaviour: model, system instructions, retrieval index, tool schema, policy bundle and application build. An incident timeline without those versions is difficult to reproduce. For sensitive actions, keep an append-only audit record of who or what approved the operation, separate from high-volume performance telemetry.

The OWASP Top 10 for Agentic Applications for 2026 is a threat-oriented checklist for deciding which control events deserve dedicated alerts. The NIST AI Risk Management Framework provides the broader governance frame: measurement should support ongoing risk management, not only a launch-time test.

A practical rollout sequence

Start with one agent workflow that has a clear owner and measurable outcome.

Map the path. List every model, retriever, tool, queue and approval step from request to outcome.
Add one trace. Propagate a correlation ID and create spans for orchestration, model calls, retrieval, tools and approvals.
Define the minimum schema. Include model, provider, version, duration, status, token counts where available, tool name, retry count and redaction status.
Choose four service-level indicators. Start with successful task rate, p95 task latency, cost per successful task and denied or escalated action rate.
Connect review to telemetry. Sample failed and high-risk traces for human analysis, then record the result as a quality or safety event.
Set retention and access rules. Decide what is never logged, who can view traces, how long data is retained and how deletion requests are handled.
Alert on patterns. Repeated tool denials, retry loops, cost spikes and quality regressions are more actionable than an alert for every failed model call.

Common mistakes

Treating token counts as quality. More context can increase cost without improving the outcome. Compare tokens with completion, correction and escalation signals.

Logging sensitive content by default. A trace that exposes secrets is an incident. Redaction and least-privilege access belong in the first instrumentation pass.

Measuring only the model. Many failures happen at the boundaries between model, context, tools and business systems. Trace the whole path.

Using one vendor dashboard as the system of record. Provider dashboards are useful, but an independent application-level trace lets a team compare providers and see non-model failures.

Sampling away the failures. Keep all high-severity, denied-action and escalation traces, and document how ordinary traces are sampled.

FAQ

What is the most important AI agent observability metric?

There is no universal winner. Start with successful task rate, p95 end-to-end latency, cost per successful outcome and a control metric such as denied or escalated actions. Together they describe reliability, performance, economics and bounded autonomy.

Should production systems log prompts and tool arguments?

Not by default. Store structured metadata and redacted summaries first. Full content should require a defined operational purpose, strict access controls, retention limits and a deletion process.

Is OpenTelemetry enough to evaluate an AI agent?

No. OpenTelemetry can standardise and correlate telemetry; it does not decide whether an answer is correct, safe or useful. Pair traces and metrics with workflow-specific evaluations, human review and outcome data.

Sources

If your team is moving an AI agent from prototype to a governed production workflow, Botchi can help design assistants and connected automations with clearer ownership, memory and operational controls.