Skip to main content

​
What is Tracing?

Tracing in Lemma provides complete visibility into your AI agent’s execution. Using OpenTelemetry, the industry-standard observability framework, Lemma captures:
  • Inputs and outputs — What goes into your agent and what comes out
  • Execution flow — The sequence of operations (LLM calls, tool invocations, database queries)
  • Timing and performance — How long each operation takes
  • Token usage — Prompt and completion tokens for model calls
  • Errors and exceptions — Where and why failures occur
Each agent execution creates a trace — a complete record from start to finish. Traces contain spans (individual operations) organized hierarchically to show how your agent works internally.

​
Core Concepts

​
wrapAgent

wrapAgent is the primary interface for tracing in Lemma. It wraps your agent function and creates a top-level OpenTelemetry span that captures the full execution — inputs, outputs, timing, and any nested operations (LLM calls, tool invocations, etc.) that happen inside it. Any OpenTelemetry-instrumented code that runs within the wrapped function automatically becomes a child span of the agent trace. This means frameworks like the Vercel AI SDK, which have built-in telemetry support, will have their spans nested under your agent’s trace with no additional setup.

​
Traces

A trace represents a single execution of your agent from start to finish. Each trace has:
  • Inputs — The initial state passed to wrapAgent
  • Outputs — The final result recorded via onComplete
  • Spans — Nested operations (LLM calls, tool invocations)
  • Timing — Duration and timing of each operation
  • Metadata — Model names, token counts, error states

​
Spans

Spans are the building blocks of a trace. Each span represents a single operation:
  • LLM generation call
  • Tool or function invocation
  • Database query
  • Custom operation
Spans are hierarchical — a parent span can contain child spans, creating a tree that shows your agent’s execution flow. The top-level span is created by wrapAgent; child spans are created automatically by instrumented frameworks or manually via the OpenTelemetry API.

​
Run ID

The run ID is Lemma’s unique identifier for a trace. It’s returned by wrapAgent and used to:
  • Link metric events to specific traces
  • Associate experiment results with test cases
  • Query and filter traces in the dashboard

​
What Gets Traced

Once tracing is set up, Lemma automatically captures:
  • Top-level agent span — Created by wrapAgent, contains inputs and outputs
  • Framework-specific spans — Automatically captured by supported integrations
  • Custom spans — Any spans you create manually with OpenTelemetry
The specific spans you see depend on your integration. See the integration guides for details:

​
Next Steps