01 Live vs indexed #
The explorer has two modes, and picking the wrong one is the usual reason a search returns nothing.
- Live (15m)
- A rolling view of the last 15 minutes, refreshed every 5 seconds. Use it while reproducing something, or when watching a deploy.
- Indexed
- A search over a chosen time range, up to the 7-day raw-span retention. Use it for anything that already happened.
You are in Live mode, which only ever shows the last 15 minutes. Switch to Indexed and pick a range.
02 Filters #
- Service
- Root service of the trace. The service-detail page's View traces button pre-fills this.
- Operation
- Root operation, usually
METHOD /route. - Min duration
- Milliseconds. The most useful single filter: set it just above your p95 to see only the slow tail.
- Errors only
- Traces containing at least one errored span.
Results show root service and operation, total duration, span count, error count, every service the trace touched, and how long ago it started.
Averages hide the requests users complain about. Take the p95 from the service page, put it in min duration, and you are looking at exactly the population that is having a bad time.
03 Reading a waterfall #
Opening a trace shows every span on a shared time axis. Horizontal position is when the span started relative to the trace; bar length is duration; indentation is parent/child depth.
- Blue bars
- Ordinary spans.
- Purple bars
- Database spans (any span carrying
db.system). - Red bars
- Spans with error status, marked with an alert icon.
- Icons
- A server icon for inbound handlers, arrows for outbound calls, a database icon for queries.
Click any span for its details: exact duration, offset from trace start, HTTP method / route / status, database system and statement, status message, all attributes, and span events (including the exception event that produced an error issue).
The three shapes worth recognising
| What you see | What it means | Where to look next |
|---|---|---|
| One long child span, parent waiting on it | The service is fine; a dependency is slow | Open the dependency's own service page |
| Many short sibling spans in sequence | N+1 pattern — a loop issuing one query or call per item | The code path that generated them |
| A long parent with little child activity | Time spent inside this service — CPU, lock contention, GC, uninstrumented work | Profiling or finer-grained spans |
A gap between a parent's start and its first child is real time the service spent before making its first outbound call — parsing, validation, auth. It is genuine work, just not separately traced.
04 What traces cannot tell you #
- They are not the source of your metrics. Rate, errors and duration come from pre-aggregated rollups computed over 100% of spans. Counting traces by hand will not reproduce the service page, and should not.
- They expire after 7 days. A chart from three weeks ago is still there; the individual traces behind it are not.
- They only show what was instrumented. An uninstrumented dependency appears as unexplained time inside its caller.