Generations

A generation is a single LLM call inside a trace. Record it with trace.recordGeneration() / trace.record_generation() after the model call completes.

Record a generation

TypeScript
Python

await lemma.trace({ name: "support-agent", input: userMessage }, async (trace) => {
  const response = await callModel(messages);

  trace.recordGeneration({
    name: "draft-reply",
    input: messages,
    output: response.text,
    model: "gpt-4o",
    llmProvider: "openai",
    llmInputMessages: messages,
    llmInvocationParameters: { temperature: 0.2 },
    usage: {
      inputTokens: response.usage?.inputTokens,
      outputTokens: response.usage?.outputTokens,
    },
  });

  return response.text;
});

def run(trace):
    response = call_model(messages)

    trace.record_generation(
        name="draft-reply",
        input=messages,
        output=response.text,
        model="gpt-4o",
        llm_provider="openai",
        llm_input_messages=messages,
        llm_invocation_parameters={"temperature": 0.2},
        input_tokens=getattr(response, "input_tokens", None),
        output_tokens=getattr(response, "output_tokens", None),
    )

    return response.text

lemma.trace("support-agent", run, input=user_message)

Pass the full ordered message list for that call as both input and llmInputMessages / llm_input_messages. The root trace input is only the current user turn; the generation should include system, prior turns, tool results, and the current user message. When the provider returns token counts, pass them as usage (TypeScript camelCase) or input_tokens / output_tokens / … kwargs (Python). Omit usage when the provider did not supply it — do not invent zeros. Explicit zeros are fine and mean “provider reported zero.” See Token usage and Analytics telemetry.

Measure a live generation

Use startGeneration() / start_generation() when you want a handle before the model call finishes. End it only after the provider returns or fails.

TypeScript
Python

const generation = trace.startGeneration({
  name: "draft-reply",
  input: messages,
  model: "gpt-4o",
});

try {
  const response = await callModel(messages);
  generation.end({
    output: response.text,
    durationMs: response.durationMs,
  });
  return response.text;
} catch (error) {
  generation.end({ error });
  throw error;
}

generation = trace.start_generation(
    name="draft-reply",
    input=messages,
    model="gpt-4o",
)

try:
    response = call_model(messages)
    generation.end(
        output=response.text,
        duration_ms=response.duration_ms,
    )
    return response.text
except Exception as error:
    generation.end(error=error)
    raise

What the SDK records

Field	Client sends	Ingest normalizes to
Generation kind	`type: "generation"`	`openinference.span.kind: "llm"`
Prompt / input	`input`, `llmInputMessages`	`input.value`, `llm.input_messages.*`
Completion / output	`output`	`output.value`
Model	`model`	`gen_ai.request.model`, `ai.model.id`
Provider	`llmProvider`	`llm.provider`, `gen_ai.system`
Token usage	`usage` / `input_tokens`… (omit when unknown)	`usage.`, `gen_ai.usage.`, `llm.token_count.*`
Duration	`durationMs` / `duration_ms`	Generation speed (distinct from root latency)
Contract props	Native props such as `llmInvocationParameters` and `llmPromptTemplate`; raw `attributes` for escape-hatch keys	Flattened OpenInference-style attributes

See Native contract props and Client fields vs ingest normalization.

Errors

If a model call fails, record a failed generation before rethrowing:

TypeScript
Python

try {
  const response = await callModel(messages);
  trace.recordGeneration({ name: "draft-reply", input: messages, output: response.text });
  return response.text;
} catch (error) {
  trace.recordGeneration({
    name: "draft-reply",
    input: messages,
    status: "ERROR",
    error,
  });
  throw error;
}

try:
    response = call_model(messages)
    trace.record_generation(
        name="draft-reply",
        input=messages,
        output=response.text,
    )
    return response.text
except Exception as error:
    trace.record_generation(
        name="draft-reply",
        input=messages,
        status="ERROR",
        error=error,
    )
    raise

Getting Started

Instrumentation

Integrations

Connections

Record a generation

Measure a live generation

What the SDK records

Errors

Next steps

Tool calls

Spans

​Record a generation

​Measure a live generation

​What the SDK records

​Errors

​Next steps

Tool calls

Spans

Record a generation

Measure a live generation

What the SDK records

Errors

Next steps