What is LLM tracing?
LLM tracing records one request to an LLM app as a tree of timed spans, one per model call, retrieval or tool call, so you can see what each step did.
LLM tracing is recording one request to an LLM application as a trace: a tree of timed spans, one for each model call, retrieval step or tool call, linked so you can follow the request from input to answer. The OpenTelemetry primer defines a trace as the path a single request takes through a system, and a span as “a single unit of work or operation.”
Tracing is the recording step. What teams do with the traces, such as scoring and alerting, is LLM observability. Tracing a multi-step agent is covered in agent tracing, and the span and attribute names are in OpenTelemetry for LLM apps.
Trace, span, log: what each one is
| Term | What it is | Tied to one request? |
|---|---|---|
| Trace | The path of one request through the system | Yes |
| Span | One unit of work inside a trace, with timing and attributes | Yes, through its trace |
| Log | A timestamped message from a service or component | Not necessarily |
The definitions come from the OpenTelemetry observability primer. The practical difference for an LLM app: a list of logged model calls tells you what was sent and received, but not which call belonged to which user request, or which step called which. A trace keeps the parent and child links. For how trace IDs and parent links work across services, see distributed tracing.
Tracing tools name these pieces differently. HoneyHive’s tracing concepts call the base unit an event, with types model (an LLM API request), tool (an external service or function call) and chain (a logical grouping of child events), and say that in other tools a session maps to a trace and an event to a span.
What one model-call span covers
As of November 2026 the OpenTelemetry GenAI conventions are still marked Development, so details can change. The inference span spec says what a model-call span should capture, and three points are easy to miss:
- Retries sit inside one span. The spans spec says a span covers the logical operation with all its retries, and the inference spec adds that time to first chunk “SHOULD include authentication, failed attempts, and retry backoff.” A slow call can therefore be a retry problem, not a slow model.
- Streaming adds a measurement. Time to first chunk is only set for streaming requests, which the spec marks with
gen_ai.request.stream. See time to first token. - Failures keep partial output.
error.typeis set when the operation ended in an error. If a stream produced output before failing, the spec says to record that partial output, and instrumentations “should not synthesize output” that never arrived.
Request settings such as gen_ai.request.temperature and gen_ai.request.max_tokens are Recommended attributes on the same span, so a trace can show which settings produced an answer.
Content is off by default
The prompt and the response are what make an LLM trace useful, and also what make it risky. The inference spec makes gen_ai.input.messages, gen_ai.output.messages and gen_ai.system_instructions Opt-In and warns that message content is “likely to contain sensitive information including user/PII data.” The spans spec says instrumentations should not capture content by default.
Decide per environment whether prompts and outputs are stored, who can read them, and what gets redacted before export.
What tracing does not do
A trace records that the model returned an answer, how long it took and what it was given. It does not say whether the answer was right. That takes an evaluator, whether a code check, an LLM judge or a person, and the usual practice is to attach the score to the trace it judged.
Tracing also does not replace metrics. A trace explains one request. Counts, rates and trends across requests come from metrics, which is why the two are used together.
Tracing in HoneyHive
HoneyHive’s tracing introduction covers capturing LLM calls, tool invocations and agent steps in a hierarchical view built on OpenTelemetry, and the quickstart instruments an OpenAI call and sends a first trace in about five minutes.
FAQ
What is the difference between LLM tracing and LLM logging?
A log is a timestamped message that, as the OpenTelemetry primer says, is not necessarily tied to any user request. A trace records the path of one request, so each model call is linked to the step that triggered it.
What is an LLM span?
One unit of work in a trace, such as a model call or a tool call. For a model call, the OpenTelemetry inference span spec suggests the name {operation} {model}, for example chat followed by the model name.
Does LLM tracing store my prompts and responses?
Only if you turn it on. The OpenTelemetry spec makes input and output messages opt-in and says instrumentations should not capture them by default, because they are likely to contain sensitive information including user and PII data.
Is LLM tracing the same as LLM evaluation?
No. A trace records what happened. Evaluation scores whether the output was good. Scores are usually attached to the trace they judge: see LLM evaluation.