OpenTelemetry for LLM apps

OpenTelemetry for LLM apps means tracing model calls, agent runs and tool calls with the standard OpenTelemetry SDKs, and naming the spans and attributes with the GenAI semantic conventions so any backend can read them. Those conventions are still in Development status, so treat the names below as a moving target.

OpenTelemetry supplies the transport: spans, OTLP and the Collector. The GenAI conventions supply the vocabulary. For why you would trace an LLM app at all, see LLM observability.

Where the conventions live

The old pages on opentelemetry.io now say the GenAI conventions “have moved” to a separate repository. Its README says it covers spans, metrics and events for GenAI clients, MCP and provider-specific conventions such as OpenAI. Its index is marked Development and links documents for inference, embeddings, tool execution, retrieval, agent invocations, events, exceptions and metrics. Provider documents exist for Anthropic, AWS Bedrock, Azure AI Inference and OpenAI, plus protocol documents for A2A and MCP.

Check the repository, not older copies, before you copy an attribute name.

The spans

The inference spec sets the model-call name and kind, and the tool spec sets the tool-call row. The agent spans spec defines the rest. It also lets a framework use invoke_{framework_entrypoint} when the entrypoint is neither an agent nor a workflow, with invoke_runner as its example, and says to report plan only when instrumentation can reliably tell planning from generic reasoning.

A nested trace of one agent request therefore looks like an agent span containing model-call spans and tool spans. That is the same parent and child structure described in distributed tracing and agent tracing.

The spans document lists the well-known operation names, including embeddings, retrieval, create_agent and several memory operations.

Attributes and what is on by default

Source: the inference spec.

Message content is the sensitive part. The spec says the message attributes are “likely to contain sensitive information including user/PII data,” and marks system instructions and tool definitions as possibly sensitive too. Instrumentations may filter or truncate messages. On tool spans, the spec makes gen_ai.tool.call.arguments and gen_ai.tool.call.result Opt-In for the same reason.

So an instrumentation that leaves Opt-In attributes off records timing, model names and token counts, and you decide separately whether prompts, outputs and tool payloads leave your process.

Metrics

The metrics document defines duration histograms, in seconds, for client operation duration (gen_ai.client.operation.duration), server request duration, time per output token and time to first token. For agents it adds gen_ai.invoke_agent.duration, gen_ai.invoke_agent.inference_calls and gen_ai.invoke_agent.tool_calls. The last two count model and tool calls per agent run, which is a direct way to see a run that loops. See time to first token for what that latency measures.

Checks before you adopt them

• Status. Development status means breaking renames are allowed. Record which version of the conventions each service emits.
• Mixed conventions. Instrumentation libraries from different projects may emit different attribute names for the same idea. HoneyHive’s mapping reference exists because of this: it maps several ecosystems onto one set of keys and says its OTel GenAI columns are pinned to semantic conventions v1.36, with later attributes such as gen_ai.provider.name (v1.37) noted inline.
• Content capture. Decide per environment whether messages and tool payloads are recorded, and who can read them.
• Gaps. The same HoneyHive reference notes that agent handoff context has no standard attribute in the ecosystems it maps, so you set it yourself as custom metadata.

Sending the data to HoneyHive

HoneyHive accepts OTLP over HTTP. The Collector guide gives the traces endpoint path /opentelemetry/v1/traces on your deployment’s API host, an Authorization: Bearer header with an ingestion key, and says to use the otlp_http exporter rather than the gRPC otlp one. HoneyHive’s tracing introduction covers the rest of the setup.

FAQ

Are the OpenTelemetry GenAI semantic conventions stable?

No. Every page in the GenAI conventions docs carries Development status, so attribute and span names can still change. Pin the version your instrumentation targets.

Do OpenTelemetry LLM spans include prompts and responses?

Not by default. The inference span spec makes gen_ai.input.messages and gen_ai.output.messages Opt-In and warns they are likely to contain sensitive information including user and PII data.

What is the difference between invoke_agent and invoke_workflow?

The agent spans spec uses invoke_agent when an LLM drives control flow through tool calls, and invoke_workflow when the application defines the control flow.

Where did the GenAI conventions move?

The old OpenTelemetry spec pages say they moved to the GenAI semantic conventions repository, which is where the current definitions live.

Can I send OpenTelemetry traces from an LLM app to HoneyHive?

Yes. HoneyHive's Collector guide describes sending OTLP over HTTP with a bearer ingestion key.