Traces

A trace is one end-to-end agent execution, from the user’s input to the final response. Everything your agent does inside that execution (LLM calls, tool calls, retrieval, app logic) is recorded as child work inside the trace.

One agent execution = one trace. Use lemma.trace() as the boundary around the run.

The root trace

TypeScript
Python

import { Lemma } from "@uselemma/tracing";

const lemma = new Lemma();

const answer = await lemma.trace(
  {
    name: "support-agent",
    input: userMessage,
  },
  async (trace) => {
    const answer = await runAgent(userMessage, trace);
    return answer;
  },
);

from uselemma_tracing import Lemma

lemma = Lemma()

async def run(trace):
    answer = await run_agent(user_message, trace)
    return answer

answer = await lemma.async_trace(
    "support-agent",
    run,
    input=user_message,
)

The SDK records the trace input, the returned value as output, and the trace name as the agent name.

Trace handles and span handles

Use the callback form when one function owns the whole run. In TypeScript, use a trace handle when work is coordinated across several helpers and you want to pass IDs around explicitly. In Python, keep the root trace in lemma.trace() / lemma.async_trace() and use span, tool, or generation handles inside the callback.

TypeScript
Python

const trace = lemma.trace({
  name: "support-agent",
  input: userMessage,
  threadId,
  userId,
});

const span = trace.startSpan("retrieve-context");
const docs = await searchDocs(userMessage);
span.recordTool({
  name: "search_docs",
  input: { query: userMessage },
  output: docs,
});
span.end({ output: { count: docs.length } });

await trace.end({ output: "Here is what I found...", durationMs: 1234 });

def run(trace):
    span = trace.start_span(name="retrieve-context")
    docs = search_docs(user_message)
    tool = trace.start_tool(
        name="search_docs",
        input={"query": user_message},
        parent_id=span.id,
    )
    tool.end(output=docs)
    span.end(output={"count": len(docs)})
    trace.output("Here is what I found...")

lemma.trace(
    "support-agent",
    run,
    input=user_message,
    thread_id=thread_id,
    user_id=user_id,
    duration_ms=1234,
)

The TypeScript trace handle has a stable trace.id. Call trace.end({ output, durationMs }) once when the run is finished to send the complete trace. Callback traces in both SDKs measure total trace duration automatically; pass durationMs / duration_ms only when you already measured it.

Root latency vs generation duration

Analytics treats these as distinct quantities:

Quantity	Source	Meaning
Root / run latency	Trace `started_at` → `ended_at` / `duration_ms`	End-to-end agent execution time
Generation duration	Generation span `duration_ms` (or its `started_at` → `ended_at`)	Time for a single model call

Keep generation timing as measured wall-clock (or provider-reported response time when that is what the integration records). Do not substitute root latency for generation speed, or sum child durations into the root when the root already has an accurate wall-clock measurement. Framework integrations already set both. For manual instrumentation, pass durationMs on each generation and let the root measure from lemma.trace() start to end() / callback return.

Record by ID

Helpers can attach work to a trace when they only have IDs from the caller. This detached helper API is TypeScript-specific.

const trace = lemma.trace();

const span = lemma.startSpan({
  traceId: trace.id,
});

lemma.recordTool({
  traceId: trace.id,
  parentSpanId: span.id,
  name: "tool call",
});

await trace.end();

Detached handle calls require traceId. If a detached span, generation, or tool belongs under a parent span, pass parentSpanId; calls that cannot attach safely warn and no-op.

Send a trace you built yourself

lemma.trace() assumes the client owns the trace lifecycle within a single process. When the producer lives elsewhere — a cross-process buffer, a queue worker, a batch backfill — build a TraceContext yourself and deliver it with ingest(). Give it a stable id so retries address the same trace.

TypeScript
Python

import { Lemma, TraceContext } from "@uselemma/tracing";

const lemma = new Lemma();

const context = new TraceContext({
  id: turnId, // stable id for this execution (use for retries)
  name: "support-agent",
  input: prompt,
  threadId: conversationId,
});
context.recordTool({ name: "search_docs", input: query, output: docs, durationMs: 25 });
context.recordGeneration({ name: "answer", model: "gpt-4o", output: finalAnswer });
context.output(finalAnswer);

await lemma.ingest(context, { startedAt });

from uselemma_tracing import Lemma, TraceContext

lemma = Lemma()

context = TraceContext(
    id=turn_id,  # stable id for this execution (use for retries)
    name="support-agent",
    input=prompt,
    thread_id=conversation_id,
)
context.record_tool(name="search_docs", input=query, output=docs, duration_ms=25)
context.record_generation(name="answer", model="gpt-4o", output=final_answer)
context.output(final_answer)

lemma.ingest(context, started_at=started_at)

ingest() POSTs one payload. Deliver one complete trace when the execution (agent turn) finishes: root input/output, thread/user, and all child spans in one call. This is required — patching a trace over time is not currently supported. ingest() is not an incremental merge API: omitted root fields do not preserve prior values, and after Lemma processes the trace once, a later re-delivery does not re-run issue extraction (occasional late child spans may still append to the tree for display). Retries of the same complete payload are safe — already-stored span IDs are skipped — so a failed send can be retried as-is. It raises on a non-2xx response and never mutates the trace’s status.

Across queues and workers, assemble the full tree in a coordinator and ingest() once when the turn is done. Do not stream partial deliveries under one trace.id.

Override output or record errors

Return values are captured automatically. Use trace.output() only when the recorded output should differ from the return value.

TypeScript
Python

await lemma.trace({ name: "support-agent", input: userMessage }, async (trace) => {
  try {
    const response = await runAgent(userMessage);
    trace.output(response.text);
    return response;
  } catch (error) {
    trace.fail(error);
    throw error;
  }
});

def run(trace):
    try:
        response = run_agent(user_message)
        trace.output(response.text)
        return response
    except Exception as error:
        trace.fail(error)
        raise

lemma.trace("support-agent", run, input=user_message)

If the callback throws, the SDK records the trace as failed, sends it to Lemma, and rethrows the original error.

Pass trace context to helpers

Pass the trace context into nested helper functions that need to record child work:

TypeScript
Python

import type { TraceContext } from "@uselemma/tracing";

function recordToolResult(trace: TraceContext, result: unknown) {
  trace.recordTool({
    name: "search_docs",
    output: result,
  });
}

from uselemma_tracing import TraceContext

def record_tool_result(trace: TraceContext, result):
    trace.record_tool(
        name="search_docs",
        output=result,
    )

Avoid relying on ambient trace state. A process can coordinate multiple traces at once, so helpers should receive the trace or span handle they need explicitly.

Add the work inside

Generations for LLM calls (model, prompt, completion).
Tool calls for tool invocations (name, args, result).
Spans for everything else (retrieval, ranking, app logic).

support-agent              <- trace root (input, output)
|- draft-reply             <- generation
|- search_docs             <- tool call
`- final-answer            <- generation

Next steps

Generations

Capture LLM calls with model, prompt, completion, and timing.

Tool calls

Record tool arguments and results.

Threads & context

Group conversations and attach users.

Trace contract

The exact shape Lemma reads.

Getting Started

Instrumentation

Integrations

Connections

The root trace

Trace handles and span handles

Root latency vs generation duration

Record by ID

Send a trace you built yourself

Override output or record errors

Pass trace context to helpers

Add the work inside

Next steps

Generations

Tool calls

Threads & context

Trace contract

​The root trace

​Trace handles and span handles

​Root latency vs generation duration

​Record by ID

​Send a trace you built yourself

​Override output or record errors

​Pass trace context to helpers

​Add the work inside

​Next steps

Generations

Tool calls

Threads & context

Trace contract

The root trace

Trace handles and span handles

Root latency vs generation duration

Record by ID

Send a trace you built yourself

Override output or record errors

Pass trace context to helpers

Add the work inside

Next steps