> ## Documentation Index
> Fetch the complete documentation index at: https://docs.uselemma.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent traces

> Create a Lemma trace for an agent execution

A **trace** is one end-to-end agent execution, from the user’s input to the final response. Everything your agent does inside that execution (LLM calls, tool calls, retrieval, app logic) is recorded as child work inside the trace.

This page is how you **create** traces in the SDK. To search, filter, and inspect ready traces in the dashboard, see [Traces](https://docs.uselemma.ai/platform/traces.md).

> [!NOTE]
>
> **One agent execution = one trace.** Use `lemma.trace()` as the boundary around the run.

## The root trace

> [tab] TypeScript
>
> ```typescript theme={null}
> import { Lemma } from "@uselemma/tracing";
>
> const lemma = new Lemma();
>
> const answer = await lemma.trace(
>   {
>     name: "support-agent",
>     input: userMessage,
>   },
>   async (trace) => {
>     const answer = await runAgent(userMessage, trace);
>     return answer;
>   },
> );
> ```

> [tab] Python
>
> ```python theme={null}
> from uselemma_tracing import Lemma
>
> lemma = Lemma()
>
> async def run(trace):
>     answer = await run_agent(user_message, trace)
>     return answer
>
> answer = await lemma.async_trace(
>     "support-agent",
>     run,
>     input=user_message,
> )
> ```

The SDK records the trace input, the returned value as output, and the trace name as the agent name.

## Trace handles and span handles

Use the callback form when one function owns the whole run. In TypeScript, use a trace handle when work is coordinated across several helpers and you want to pass IDs around explicitly. In Python, keep the root trace in `lemma.trace()` / `lemma.async_trace()` and use span, tool, or generation handles inside the callback.

> [tab] TypeScript
>
> ```typescript theme={null}
> const trace = lemma.trace({
>   name: "support-agent",
>   input: userMessage,
>   threadId,
>   userId,
> });
>
> const span = trace.startSpan("retrieve-context");
> const docs = await searchDocs(userMessage);
> span.recordTool({
>   name: "search_docs",
>   input: { query: userMessage },
>   output: docs,
> });
> span.end({ output: { count: docs.length } });
>
> await trace.end({ output: "Here is what I found...", durationMs: 1234 });
> ```

> [tab] Python
>
> ```python theme={null}
> def run(trace):
>     span = trace.start_span(name="retrieve-context")
>     docs = search_docs(user_message)
>     tool = trace.start_tool(
>         name="search_docs",
>         input={"query": user_message},
>         parent_id=span.id,
>     )
>     tool.end(output=docs)
>     span.end(output={"count": len(docs)})
>     trace.output("Here is what I found...")
>
> lemma.trace(
>     "support-agent",
>     run,
>     input=user_message,
>     thread_id=thread_id,
>     user_id=user_id,
>     duration_ms=1234,
> )
> ```

The TypeScript trace handle has a stable `trace.id`. Call `trace.end({ output, durationMs })` once when the run is finished to send the complete trace. Callback traces in both SDKs measure total trace duration automatically; pass `durationMs` / `duration_ms` only when you already measured it.

## Root latency vs generation duration

Analytics treats these as **distinct** quantities:

| Quantity | Source | Meaning |
| - | - | - |
| **Root / run latency** | Trace `started_at` → `ended_at` / `duration_ms` | End-to-end agent execution time |
| **Generation duration** | Generation span `duration_ms` (or its `started_at` → `ended_at`) | Time for a single model call |

Keep generation timing as measured wall-clock (or provider-reported response time when that is what the integration records). Do not substitute root latency for generation speed, or sum child durations into the root when the root already has an accurate wall-clock measurement.

Framework integrations already set both. For manual instrumentation, pass `durationMs` on each generation and let the root measure from `lemma.trace()` start to `end()` / callback return.

## Record by ID

Helpers can attach work to a trace when they only have IDs from the caller. This detached helper API is TypeScript-specific.

```typescript theme={null}
const trace = lemma.trace();

const span = lemma.startSpan({
  traceId: trace.id,
});

lemma.recordTool({
  traceId: trace.id,
  parentSpanId: span.id,
  name: "tool call",
});

await trace.end();
```

Detached handle calls require `traceId`. If a detached span, generation, or tool belongs under a parent span, pass `parentSpanId`; calls that cannot attach safely warn and no-op.

## Send a trace you built yourself

`lemma.trace()` assumes the client owns the trace lifecycle within a single process. When the producer lives elsewhere (a cross-process buffer, a queue worker, a batch backfill), build a `TraceContext` yourself and deliver it with `ingest()`. Give it a stable `id` so retries address the same trace.

> [tab] TypeScript
>
> ```typescript theme={null}
> import { Lemma, TraceContext } from "@uselemma/tracing";
>
> const lemma = new Lemma();
>
> const context = new TraceContext({
>   id: turnId, // stable id for this execution (use for retries)
>   name: "support-agent",
>   input: prompt,
>   threadId: conversationId,
> });
> context.recordTool({ name: "search_docs", input: query, output: docs, durationMs: 25 });
> context.recordGeneration({ name: "answer", model: "gpt-4o", output: finalAnswer });
> context.output(finalAnswer);
>
> await lemma.ingest(context, { startedAt });
> ```

> [tab] Python
>
> ```python theme={null}
> from uselemma_tracing import Lemma, TraceContext
>
> lemma = Lemma()
>
> context = TraceContext(
>     id=turn_id,  # stable id for this execution (use for retries)
>     name="support-agent",
>     input=prompt,
>     thread_id=conversation_id,
> )
> context.record_tool(name="search_docs", input=query, output=docs, duration_ms=25)
> context.record_generation(name="answer", model="gpt-4o", output=final_answer)
> context.output(final_answer)
>
> lemma.ingest(context, started_at=started_at)
> ```

`ingest()` POSTs one payload. Deliver **one complete trace** when the execution (agent turn) finishes: root input/output, thread/user, and all child spans in one call. This is required: patching a trace over time is not currently supported.

`ingest()` is not an incremental merge API: omitted root fields do not preserve prior values, and after Lemma processes the trace once, a later re-delivery does not re-run issue extraction (occasional late child spans may still append to the tree for display). Retries of the same complete payload are safe, because already-stored span IDs are skipped, so a failed send can be retried as-is. It raises on a non-2xx response and never mutates the trace’s status.

> [!WARNING]
>
> Across queues and workers, assemble the full tree in a coordinator and `ingest()` once when the turn is done. Do not stream partial deliveries under one `trace.id`.

When one user turn spans a host and a sandbox, use `startTurn` / `attachTurn` / `apply` (Python `start_turn` / `attach_turn`) so they become one root. See [Cross-process turns](https://docs.uselemma.ai/tracing/instrumentation/cross-process-turns.md).

## Override output or record errors

Return values are captured automatically. Use `trace.output()` only when the recorded output should differ from the return value.

> [tab] TypeScript
>
> ```typescript theme={null}
> await lemma.trace({ name: "support-agent", input: userMessage }, async (trace) => {
>   try {
>     const response = await runAgent(userMessage);
>     trace.output(response.text);
>     return response;
>   } catch (error) {
>     trace.fail(error);
>     throw error;
>   }
> });
> ```

> [tab] Python
>
> ```python theme={null}
> def run(trace):
>     try:
>         response = run_agent(user_message)
>         trace.output(response.text)
>         return response
>     except Exception as error:
>         trace.fail(error)
>         raise
>
> lemma.trace("support-agent", run, input=user_message)
> ```

If the callback throws, the SDK records the trace as failed, sends it to Lemma, and rethrows the original error. Automatic delivery (`trace` / `async_trace` / `TraceHandle.end()`) fails open: a Lemma ingest 4xx/5xx or network error is logged in debug mode and dropped, so it cannot fail the caller’s application or replace the original error. A transport failure on a successful run is not recorded as an agent failure. Use `ingest()` when you need a failed send to raise so you can retry.

## Pass trace context to helpers

Pass the trace context into nested helper functions that need to record child work:

> [tab] TypeScript
>
> ```typescript theme={null}
> import type { TraceContext } from "@uselemma/tracing";
>
> function recordToolResult(trace: TraceContext, result: unknown) {
>   trace.recordTool({
>     name: "search_docs",
>     output: result,
>   });
> }
> ```

> [tab] Python
>
> ```python theme={null}
> from uselemma_tracing import TraceContext
>
> def record_tool_result(trace: TraceContext, result):
>     trace.record_tool(
>         name="search_docs",
>         output=result,
>     )
> ```

Avoid relying on ambient trace state. A process can coordinate multiple traces at once, so helpers should receive the trace or span handle they need explicitly.

## Add the work inside

* [Generations](https://docs.uselemma.ai/tracing/instrumentation/generations.md) for LLM calls (model, prompt, completion).
* [Tool calls](https://docs.uselemma.ai/tracing/instrumentation/tool-calls.md) for tool invocations (name, args, result).
* [Spans](https://docs.uselemma.ai/tracing/instrumentation/spans.md) for everything else (retrieval, ranking, app logic).

```text theme={null}
support-agent              <- trace root (input, output)
├── draft-reply            <- generation
├── search_docs            <- tool call
└── final-answer           <- generation
```

## Next steps

> [card] [Generations](https://docs.uselemma.ai/tracing/instrumentation/generations.md)
>
> Capture LLM calls with model, prompt, completion, and timing.

> [card] [Tool calls](https://docs.uselemma.ai/tracing/instrumentation/tool-calls.md)
>
> Record tool arguments and results.

> [card] [Traces in the dashboard](https://docs.uselemma.ai/platform/traces.md)
>
> Search and inspect the traces you send.

> [card] [Trace contract](https://docs.uselemma.ai/reference/trace-contract.md)
>
> The exact shape Lemma reads.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.