Adding the Lemma SDK next to OpenTelemetry is fully supported. OTel instrumentation on its own is not sufficient for Lemma, because it usually does not produce the trace contract Lemma reads.
Why existing instrumentation is not enough
Lemma is an opinionated sink. It reads a specific trace shape to power input/output display, model visibility, timing, tool visibility, threads, and automated issue detection. See Overview for how Lemma reads your traces. Generic OpenTelemetry is a flexible observability layer. It will happily record spans, attributes, and timing, but it does not guarantee the things Lemma depends on:- One root trace per agent execution with the current user input and the final output or error.
- Typed children — generations, tools, and generic spans, not undifferentiated spans.
- Per-generation message history — the full ordered message list sent to each LLM call.
Concept mapping
The mental model transfers cleanly. What changes is that Lemma needs its children typed and its root complete. For the canonical field semantics and the client-vs-normalization mapping, see the trace contract.| OpenTelemetry concept | Lemma primitive | What Lemma needs |
|---|---|---|
| OTel root span | Lemma trace record (root) | One agent execution = one trace, with input and final output or error |
| Generic child span | Lemma child span | A typed entry in spans[] |
| LLM span | Lemma generation (type: "generation") | model plus the full per-call message history |
| Tool span | Lemma tool (type: "tool") | Tool name, arguments, and result or error |
| Retrieval, ranking, or app-logic span | Lemma span (type: "span") | input and output (or error) |
| Parent / child span nesting | parent_id nesting | Children under the step that caused them |
| Span start / end timestamps | started_at / ended_at / duration_ms | Real measured bounds |
type is a discriminator. Setting a model on an untyped span does not make it a generation — you must set type: "generation". The SDK typed helpers (recordGeneration / recordTool / recordSpan) do this for you.What transfers and what does not
Transfers — the structural intuition you already have:- Nesting: parent/child span trees map directly onto Lemma’s
parent_idnesting. - Timing: start/end timestamps map onto Lemma’s
started_at/ended_at/duration_ms. - One logical run: the idea that a single agent execution is one unit maps onto one Lemma trace.
- A dedicated Lemma root trace per execution (
lemma.trace(...)). - Typed generations and tools, rather than generic spans.
- Root input and output (or error) on the trace record.
- Per-generation message history — the full ordered message list, not just the latest user string.
Run Lemma side by side
Leave your OpenTelemetry instrumentation exactly where it is. Wrap the same agent execution inlemma.trace() and record the LLM calls and tools you already know about as typed Lemma children. This produces a conforming Lemma trace without disturbing your existing pipeline.
Install and configure the SDK first — see Setup.
- TypeScript
- Python
input and llmInputMessages / llm_input_messages. The root trace input is only the current user turn. See Generations and Tool calls for live handles, errors, and nesting under a measured parent span.
If your run is coordinated across streaming callbacks or helpers rather than one function, use a TypeScript trace handle and end it from the terminal callback. See Traces.
Keep your OpenTelemetry setup
The Lemma SDK delivers its trace JSON over its own HTTP path; it does not register as an exporter or span processor in your OpenTelemetry pipeline. That means your existing OTel exporters can stay untouched — the two run independently and neither depends on the other. The one place the two pipelines meet today is the Vercel AI SDK, which is built on OpenTelemetry. There you addvercelAI() to the AI SDK’s telemetry integrations alongside any exporters you already use, and it produces the Lemma trace from AI SDK lifecycle events. See Vercel AI SDK.
For frameworks with a first-class Lemma integration, prefer the integration over hand-rolling the mapping:
Outside those framework integrations, treat OTel and Lemma as separate additive layers: keep OTel for your existing telemetry and use the Lemma SDK directly for the Lemma trace contract, as shown above.
Pitfalls
| Pitfall | Fix |
|---|---|
| Assuming OTel spans already satisfy Lemma | They usually do not carry the Lemma contract. Emit it explicitly with the Lemma SDK. |
| No Lemma root trace | Wrap each agent execution in lemma.trace() so children have one root to attach to. |
| Untyped children | Record LLM calls with recordGeneration and tools with recordTool so they render and filter as generations and tools. |
| Missing message history | Put the full ordered message list on each generation’s input and llmInputMessages / llm_input_messages, not just the latest user string. |
Related pages
Setup
Install the SDK and point it at Lemma.
Trace contract
The exact shape and fields Lemma reads.
Building high-quality traces
Bad → better → best examples for the ideal path.
Integrations
Framework integrations that produce the contract for you.