Skip to main content

​
What is Tracing?

Tracing in Lemma provides complete visibility into your AI agent’s execution. Using OpenTelemetry, the industry-standard observability framework, Lemma captures:
  • Inputs and outputs — What goes into your agent and what comes out
  • Execution flow — The sequence of operations (LLM calls, tool invocations, database queries)
  • Timing and performance — How long each operation takes
  • Token usage — Prompt and completion tokens for model calls
  • Errors and exceptions — Where and why failures occur
Each agent execution creates a trace — a complete record from start to finish. Traces contain spans (individual operations) organized hierarchically to show how your agent works internally.

​
Core Concepts

​
wrapAgent

wrapAgent is the primary interface for tracing in Lemma. It wraps your agent function and creates a top-level OpenTelemetry span that captures the full execution — inputs, outputs, timing, and any nested operations (LLM calls, tool invocations, etc.) that happen inside it. Any OpenTelemetry-instrumented code that runs within the wrapped function automatically becomes a child span of the agent trace. This means frameworks like the Vercel AI SDK, which have built-in telemetry support, will have their spans nested under your agent’s trace with no additional setup. Provider SDKs (raw OpenAI, Anthropic, etc.) do not emit those child spans by default. Add OpenInference via Provider instrumentation, or hand-roll spans.

​
Traces

A trace represents a single execution of your agent from start to finish. Each trace has:
  • Inputs — The initial state passed to wrapAgent
  • Outputs — The final result recorded via onComplete
  • Spans — Nested operations (LLM calls, tool invocations)
  • Timing — Duration and timing of each operation
  • Metadata — Model names, token counts, error states

​
Spans

Spans are the building blocks of a trace. Each span represents a single operation:
  • LLM generation call
  • Tool or function invocation
  • Database query
  • Custom operation
Spans are hierarchical — a parent span can contain child spans, creating a tree that shows your agent’s execution flow. The top-level span is created by wrapAgent; child spans are created automatically by instrumented frameworks or manually via the OpenTelemetry API.

​
Run ID

The run ID is Lemma’s unique identifier for a trace. It’s returned by wrapAgent and used to:
  • Link metric events to specific traces
  • Associate experiment results with test cases
  • Query and filter traces in the dashboard

​
Thread ID

Optional lemma.thread_id links separate runs (multi-turn chats, retries) in the dashboard. Pass it via the second argument to the function returned by wrapAgent / wrap_agent (threadId / thread_id). See Wrapping your agent.

​
What Gets Traced

Once tracing is set up, Lemma automatically captures:
  • Top-level agent span — Created by wrapAgent, contains inputs and outputs
  • Framework-specific spans — Automatically captured by supported integrations
  • Custom spans — Any spans you create manually with OpenTelemetry
The specific spans you see depend on your integration. See the integration guides for details:

​
Next Steps