Concepts

Understanding these core concepts will help you get the most out of Lemma’s observability platform.

​

How a trace is structured

A single agent execution produces a tree of spans. The root span represents one request/session execution, and everything inside that execution (LLM calls, tool runs, retrieval, business logic) appears as child spans. Provider/framework instrumentation creates gen_ai.chat spans; app/tool instrumentation creates operational child spans. Lemma ingests this hierarchy and preserves parent/child relationships.

---

​

Traces

A trace represents a single execution of your agent from start to finish. It captures:

• Inputs — The initial state and parameters passed to your agent
• Outputs — The final result produced by your agent
• Spans — Nested operations within the execution (LLM calls, tool invocations, etc.)
• Timing — Duration and timing of each operation
• Metadata — Additional context like model names, token counts, and error states

Traces are built on OpenTelemetry. Each trace has a unique trace ID and contains one or more spans in a hierarchy.

​

Thread identifier

To link multi-turn conversations, include lemma.thread_id on related spans. This is the canonical thread identifier in Lemma. You can also include framework-native session fields, but lemma.thread_id should be the source of truth for thread grouping in Lemma.

​

Spans

Spans are the building blocks of a trace. Each span represents a single operation within your agent’s execution, such as:

• An LLM generation call
• A tool or function invocation
• A database query
• A custom operation you want to track

Spans are hierarchical — a parent span can contain multiple child spans, creating a tree structure that represents your agent’s execution flow. This hierarchy makes it easy to understand:

• Which operations happened in what order
• How long each operation took
• Where errors occurred in the execution path

​

Projects

A project is the top-level container in Lemma. It groups:

• All traces from your agent(s)

Each project has:

• A project ID — Used when sending traces and making API calls
• An API key — For authentication
• A dashboard for viewing and analyzing data

Most organizations use one project per application or product, but you can create multiple projects to separate environments (dev/staging/prod) or different agent types.

​

Tracer Provider

The tracer provider is the OpenTelemetry component responsible for:

• Creating and managing spans
• Exporting trace data to Lemma’s OTLP endpoint
• Handling batching and retries

For greenfield setups, the recommended path is to configure a LangfuseSpanProcessor and give it an OpenTelemetry exporter that targets Lemma. If your app already emits OpenTelemetry-compatible spans, keep that instrumentation and add Lemma as an OpenTelemetry export destination.

​

Next Steps

Now that you understand Lemma’s core concepts:

• Explore Tracing to start sending traces
• Start with Langfuse integration for implementation