Manual Instrumentation

Use manual instrumentation when you have a custom agent framework, need precise control over every span, or want to add structured child spans that OpenInference does not emit automatically. For setup (registerOTel()) and the run boundary (agent()), see Overview and Quickstart.

Span helpers (recommended)

The @uselemma/tracing and uselemma_tracing packages export typed helpers that wrap functions with child spans automatically. Use these instead of raw startActiveSpan calls — they handle error recording, span lifecycle, and automatic I/O capture for you.

Helper	`span.type`	Use for
`trace(name, fn)`	(none)	General-purpose child span
`tool(name, fn)`	`tool`	Tool / function execution
`llm(name, fn)`	`generation`	LLM call (when OpenInference is not used)
`retrieval(name, fn)`	`retriever`	Vector search, document retrieval

All helpers create child spans under the currently active context — they automatically nest under the enclosing agent() span without any extra wiring. Automatic I/O capture: every helper serializes the function’s input as input.value and its return value as output.value on the span. These appear as “Input” and “Output” in the Lemma trace view without any manual setAttribute calls. In TypeScript, helpers are always called as tool("name", fn). In Python, they also work as decorators (@tool("name")).

TypeScript
Python

import { agent, tool, llm, retrieval, trace } from "@uselemma/tracing";

const search = retrieval("vector-search", async (query: string) => {
  return vectorDB.search(query, { topK: 5 });
});

const lookup = tool("lookup-order", async (orderId: string) => {
  return db.orders.findById(orderId);
});

const generate = llm("gpt-4o", async (prompt: string) => {
  return openai.chat.completions.create({ model: "gpt-4o", messages: [...] });
});

const formatOutput = trace("format-output", async (raw: string) => raw.trim());

const myAgent = agent("my-agent", async (input: string) => {
  const docs = await search(input);         // span: vector-search  (span.type: retriever)
  const order = await lookup("123");         // span: lookup-order   (span.type: tool)
  const response = await generate(input);   // span: gpt-4o         (span.type: generation)
  return formatOutput(response.text);        // span: format-output
});

All helpers support a decorator form and a wrapper call form — use whichever fits your code style.

from uselemma_tracing import agent, tool, llm, retrieval, trace, TraceContext

# --- decorator form ---
@retrieval("vector-search")
async def search(query: str) -> list:
    return await vector_db.search(query)

@tool("lookup-order")
async def lookup_order(order_id: str) -> dict:
    return await db.orders.get(order_id)

# --- wrapper call form (equivalent) ---
async def _generate(prompt: str) -> str:
    response = await openai.chat.completions.create(
        model="gpt-4o", messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

generate = llm("gpt-4o", _generate)        # span: gpt-4o (span.type: generation)
format_output = trace("format-output", str.strip)

async def run_agent(user_message: str, ctx: TraceContext) -> str:
    docs = await search(user_message)          # span: vector-search  (span.type: retriever)
    order = await lookup_order("123")           # span: lookup-order   (span.type: tool)
    response = await generate(user_message)    # span: gpt-4o         (span.type: generation)
    return format_output(response)              # span: format-output

myAgent = agent("my-agent", run_agent)

LLM step spans (raw OTel)

For full control over step-level attributes (model, tokens, cost, finish reason), create child spans manually using the OpenTelemetry API:

TypeScript
Python

import { trace } from "@opentelemetry/api";
import { agent } from "@uselemma/tracing";

const tracer = trace.getTracer("my-agent");

const wrapped = agent("support-agent", async (input: string) => {
  const answer = await tracer.startActiveSpan("llm.step.generate", async (stepSpan) => {
    const response = await llmCall(input);
    stepSpan.setAttribute("llm.model.requested", "gpt-4o");
    stepSpan.setAttribute("llm.tokens.prompt_uncached", 320);
    stepSpan.setAttribute("llm.tokens.completion", 140);
    stepSpan.setAttribute("llm.finish_reason", "stop");
    stepSpan.end();
    return response;
  });

  return answer;
});

from opentelemetry import trace as otel_trace
from uselemma_tracing import agent, TraceContext

tracer = otel_trace.get_tracer("my-agent")

async def run_agent(user_message: str, ctx: TraceContext) -> str:
    with tracer.start_as_current_span("llm.step.generate") as step_span:
        response = await llm_call(user_message)
        step_span.set_attribute("llm.model.requested", "gpt-4o")
        step_span.set_attribute("llm.tokens.prompt_uncached", 320)
        step_span.set_attribute("llm.tokens.completion", 140)
        step_span.set_attribute("llm.finish_reason", "stop")
    return response

If you use OpenInference instrumentation for your provider, these attributes are emitted automatically.

Tool call spans (raw OTel)

TypeScript
Python

async function callWeatherTool(city: string) {
  return tracer.startActiveSpan("tool.call", async (toolSpan) => {
    toolSpan.setAttribute("tool.name", "get_weather");
    toolSpan.setAttribute("tool.args", JSON.stringify({ city }));
    try {
      const result = await getWeather(city);
      toolSpan.setAttribute("tool.result", JSON.stringify(result));
      toolSpan.end();
      return result;
    } catch (error) {
      toolSpan.recordException(error as Error);
      toolSpan.setAttribute("tool.status", "error");
      toolSpan.end();
      throw error;
    }
  });
}

async def call_weather_tool(city: str):
    with tracer.start_as_current_span("tool.call") as tool_span:
        tool_span.set_attribute("tool.name", "get_weather")
        tool_span.set_attribute("tool.args", f'{{"city":"{city}"}}')
        try:
            result = await get_weather(city)
            tool_span.set_attribute("tool.result", str(result))
            return result
        except Exception as err:
            tool_span.record_exception(err)
            tool_span.set_attribute("tool.status", "error")
            raise

For most tool-call use cases, the tool() helper above is simpler — use raw startActiveSpan only when you need explicit control over attributes like tool.args and tool.result.

Next Steps

Custom attributes — attach user ID, session, and environment metadata to run spans
Multi-turn threads — link related runs into a conversation thread
Troubleshooting — spans not appearing, nesting issues
From-Scratch Agent recipe — framework-free step-by-step instrumentation walkthrough
Adding provider instrumentation — auto-instrument your provider SDK so gen_ai.chat child spans appear without manual step spans

Getting Started

Tracing

Connections

Manual Instrumentation

Span helpers (recommended)

LLM step spans (raw OTel)

Tool call spans (raw OTel)

Next Steps

Getting Started

Tracing

Connections

​Span helpers (recommended)

​LLM step spans (raw OTel)

​Tool call spans (raw OTel)

​Next Steps

Span helpers (recommended)

LLM step spans (raw OTel)

Tool call spans (raw OTel)

Next Steps