Instrument an agent

This page starts from an untraced support agent and adds Lemma software development kit (SDK) calls one piece at a time. Complete the Quickstart first so the SDK is installed and a Lemma client can read your credentials. The per-primitive pages go deeper on each piece: Agent traces, Generations, Tool calls, Spans, and Threads and context. One agent execution is one lemma.trace() call. Record model calls, tool calls, and app work inside that trace. If that execution spans a host and a sandbox, follow Cross-process turns instead of starting a second root.

What you will build

support-agent                  <- trace root (input, output, agent name, thread, user)
├── retrieve-context           <- span (retrieval work)
├── search_docs                <- tool call (query, docs)
├── lookup_order               <- tool call (user ID, order)
└── answer                     <- generation (model, prompt, completion)

Before you start

The examples assume you already have an agent shaped like this:

TypeScript
Python

async function handleSupportRequest(req: {
  message: string;
  conversationId: string;
  userId: string;
}): Promise<string> {
  const docs = await searchDocs(req.message);
  const order = await lookupOrder(req.userId);
  const messages = buildPrompt(req.message, docs, order);
  const response = await callModel(messages);

  return response.text;
}

def handle_support_request(message: str, conversation_id: str, user_id: str) -> str:
    docs = search_docs(message)
    order = lookup_order(user_id)
    messages = build_prompt(message, docs, order)
    response = call_model(messages)

    return response.text

Steps

Wrap the whole agent run in a trace

Start with the root boundary. The returned value becomes the trace output.

TypeScript
Python

import { Lemma } from "@uselemma/tracing";

const lemma = new Lemma();

return lemma.trace(
  {
    name: "support-agent",
    input: req.message,
    threadId: req.conversationId,
    userId: req.userId,
  },
  async (trace) => {
    const docs = await searchDocs(req.message);
    const order = await lookupOrder(req.userId);
    const messages = buildPrompt(req.message, docs, order);
    const response = await callModel(messages);

    return response.text;
  },
);

from uselemma_tracing import Lemma

lemma = Lemma()

def run(trace):
    docs = search_docs(message)
    order = lookup_order(user_id)
    messages = build_prompt(message, docs, order)
    response = call_model(messages)

    return response.text

return lemma.trace(
    "support-agent",
    run,
    input=message,
    thread_id=conversation_id,
    user_id=user_id,
)

Add a span around retrieval work

Use a span for app work that is not itself a model call or tool call.

TypeScript
Python

const retrieve = trace.startSpan({
  name: "retrieve-context",
  input: { query: req.message },
});

const docs = await searchDocs(req.message);

retrieve.end({
  output: { count: docs.length },
  durationMs: 120,
});

retrieve = trace.start_span(
    name="retrieve-context",
    input={"query": message},
)

docs = search_docs(message)

retrieve.end(
    output={"count": len(docs)},
    duration_ms=120,
)

Pass durationMs / duration_ms when you already measured the retrieval time. If you omit it, the SDK records timestamps and Lemma can allocate missing child durations from the parent.

Record each tool call

Record tools after they return so you can include both arguments and results.

TypeScript
Python

trace.recordTool({
  name: "search_docs",
  input: { query: req.message },
  output: docs,
  durationMs: 45,
  toolParameters: { query: "string" },
});

const order = await lookupOrder(req.userId);
trace.recordTool({
  name: "lookup_order",
  input: { userId: req.userId },
  output: order,
  durationMs: 30,
});

trace.record_tool(
    name="search_docs",
    input={"query": message},
    output=docs,
    duration_ms=45,
    tool_parameters={"query": "string"},
)

order = lookup_order(user_id)
trace.record_tool(
    name="lookup_order",
    input={"user_id": user_id},
    output=order,
    duration_ms=30,
)

Use stable, boring names like search_docs, lookup_order, and rerank_results. They make traces easier to scan.

Record the model call as a generation

Generations are for LLM calls. Include the model, prompt/input, output, and timing when available.

TypeScript
Python

const messages = buildPrompt(req.message, docs, order);
const response = await callModel(messages);

trace.recordGeneration({
  name: "answer",
  model: response.model,
  input: messages,
  output: response.text,
  durationMs: response.durationMs,
});

return response.text;

messages = build_prompt(message, docs, order)
response = call_model(messages)

trace.record_generation(
    name="answer",
    model=response.model,
    input=messages,
    output=response.text,
    duration_ms=response.duration_ms,
)

return response.text

Record root trace duration when the run is done

Callback traces are measured automatically when the callback returns. For trace handles, record root duration when you call trace.end(...).

TypeScript
Python

const trace = lemma.trace({ name: "support-agent", input: req.message });
const response = await runAgent(trace);

await trace.end({
  output: response.text,
  durationMs: totalDurationMs,
});

# Callback traces are measured automatically.
# Pass duration_ms only when you already have an external measurement.
return lemma.trace(
    "support-agent",
    run,
    input=message,
    duration_ms=total_duration_ms,
)

For open handles, leave duration blank until the work is actually finished. The SDK cannot know the final elapsed time until span.end(...), trace.end(...), or the callback returns.

Mark failures where you catch them

lemma.trace() records uncaught errors on the root trace. If you catch a tool or model error before re-raising, record that child failure too.

TypeScript
Python

try {
  const order = await lookupOrder(req.userId);
  trace.recordTool({
    name: "lookup_order",
    input: { userId: req.userId },
    output: order,
  });
} catch (error) {
  trace.recordTool({
    name: "lookup_order",
    input: { userId: req.userId },
    error,
  });
  throw error;
}

try:
    order = lookup_order(user_id)
    trace.record_tool(
        name="lookup_order",
        input={"user_id": user_id},
        output=order,
    )
except Exception as error:
    trace.record_tool(
        name="lookup_order",
        input={"user_id": user_id},
        error=error,
    )
    raise

Go deeper

Use these pages for field-level detail and framework-specific tracing.

Trace contract

The exact fields and native SDK props Lemma reads.

Runnable examples

The same docs chat agent, instrumented once per stack.

Vercel AI SDK

Record AI SDK model calls and tool executions automatically.

Troubleshooting

Fix missing traces, empty outputs, and shape issues.

Tracing

Integrations

Troubleshooting

Reference

Instrument an agent

What you will build

Before you start

Steps

Go deeper

Trace contract

Runnable examples

Vercel AI SDK

Troubleshooting

​What you will build

​Before you start

​Steps

​Go deeper

Trace contract

Runnable examples

Vercel AI SDK

Troubleshooting

What you will build

Before you start

Steps

Go deeper