A single LLM call is straightforward to instrument: you record the start time, the end time, the model, the token counts. The call is a leaf node. When something goes wrong, you look at the call.
Multi-step LLM workflows are different. A research pipeline might involve query rewriting, embedding, retrieval, reranking, context assembly, generation, and response validation -- seven steps, some of which may themselves fan out into parallel sub-calls. When a response is wrong or slow or expensive, the answer could be in any of those steps. Without structured tracing, you are guessing.
The problem with event logs for chained workflows
The first instrumentation approach most teams try for complex workflows is event logging: add a log line at the beginning and end of each step with a timestamp and some metadata. This produces a chronological log of what happened, which can be grepped and analyzed. It has a fundamental limitation: it doesn't capture the parent-child relationships between steps.
When your workflow produces a sub-call as part of a step, the log has two independent entries with timestamps but no structural relationship. You can sometimes reconstruct the relationship from timestamps and a shared request ID, but this is fragile. If steps run in parallel, the log interleaving makes reconstruction difficult or impossible.
The span model solves this by requiring each span to carry a parent ID. Every span in a trace can be reconstructed into a tree by following parent IDs. Parallel steps become sibling spans with the same parent. Sub-calls become child spans of the step that initiated them. The structure is captured in the data, not inferred after the fact from timestamps.
Modeling a multi-step workflow as a flat span list
The trace model is a tree, but what you actually store and query is a flat list of spans with parent references. This matters for implementation: you don't need to build a recursive data structure to create traces. You need to ensure that each span carries its parent ID when it is created.
Here is how we model a typical RAG pipeline in Spanloom's span model:
- Root span:
rag_pipeline-- the overall request. Contains all metadata that applies to the full request: user ID, session ID, pipeline version, total latency, total cost. - Child span:
query_rewrite-- the step that rewrites the user query for retrieval. Contains original query, rewritten query, model used, token counts. - Child span:
retrieve_docs-- the retrieval step. Contains query embedding, top-K count, retrieved chunk IDs, retrieval latency. - Child span:
rerank-- optional reranking of retrieved chunks. Contains pre-rank and post-rank chunk lists, reranker model, latency. - Child span:
generate-- the main LLM call. Contains input token count, output token count, model, prompt version, cost. - Child span:
validate_response-- any post-processing validation. Contains validation result, fallback flag.
Each span has its own start/end timestamps, status, and attributes. The root span's duration is the wall time for the full request. The sum of child span durations shows you where time is actually being spent.
Context propagation across service boundaries
Multi-step workflows often cross service boundaries. Your retrieval step might call a vector search service. Your generation step might go through an inference gateway. Context propagation is the mechanism that links spans across these boundaries.
In OpenTelemetry, context is propagated via headers (W3C TraceContext: traceparent and tracestate). When you make an HTTP call to another service, you inject the current trace context into the request headers. The receiving service extracts the context and creates its spans as children of the span that made the call.
If your downstream services are instrumented with OTel, this happens automatically. If they are not, you can still propagate your own context manually: pass the trace ID and parent span ID in your request and extract them on the receiving end to attach the child span to the right parent in your own trace storage.
What to look for in the resulting trace
A trace for a well-instrumented multi-step workflow tells you three things immediately:
Where the latency is. The visual timeline of the trace shows which spans are sequential, which are parallel, and which ones dominate the wall time. If your total latency is 3.2 seconds and the retrieve_docs span is 2.1 seconds, that's your optimization target -- not the model.
Where the cost is. Each span that touches a model records its token counts and cost. A multi-step workflow that includes a query rewrite call is spending money on that rewrite; the trace makes this visible. If query rewriting costs as much as generation on simple queries, you might not need it for those cases.
Where the failures are. A span with status ERROR or with a fallback: true attribute identifies exactly which step in the workflow failed. You don't need to correlate log lines; the span itself tells you.
At high volume, you can aggregate across traces: what is the 95th percentile latency for retrieve_docs? What fraction of traces have a fallback: true span? What is the average cost of traces that include a query rewrite vs. those that don't? These aggregations are where multi-step workflow observability pays off at scale -- not in the individual trace, but in the patterns across thousands of them.