LangChain
- Root output: Reasoning content blocks become the answer string on the root trace output. The generation span keeps the full block list. Tool-call payloads stay unchanged.
LangChain
- Model name: When a chat model is not serializable, the handler reads
ls_model_namefrom run metadata after invocation params.
Tool calls
- Failure payloads: A tool result with
success: falseorstatusoferrororfailedis recorded as an error. The span usesstructuredContent.errorwhen that field is set.
LangChain
- Callback handler:
LemmaLangChainCallbackHandleraccepts LangChain 1.x callback arguments and setsrun_inline,raise_error, and theignore_*flags.
Vercel AI
- Telemetry types: The Vercel AI telemetry object is assignable to the AI SDK
Telemetrytype without a cast.
Mastra
- Exporter types:
exportTracingEventaccepts the framework event shape, so you can pass the exporter without a cast.
Generations
- Model identity: A typed generation span copies the model from the invocation, the response, or response metadata when params omit it. Completion text is not used as a model id.
Vercel AI
- Provider system: A generation records the provider system when the call sets
messages.
Tracing
- Coding-agent turns:
codingAgentTurnTracebuilds one trace from a completed coding-agent turn: prompt, tool calls, and response.
Generations
- Token usage: LangChain, OpenAI Agents, Vercel AI SDK, and Mastra record token usage, including cache and reasoning tokens from Agents SDK and AI SDK 7 shapes.
- Provider: Generation spans set the provider system from the model provider.
Tracing
- Release: Pass
releaseto the client, or setLEMMA_RELEASE, to stamp the running revision on each trace. - Provenance: Spans record the SDK language and integration so missing usage is distinct from a real zero.
Tracing
- Inputs and outputs: Prompts, tool inputs, tool outputs, model output, and error messages are always recorded. The
recordInputsandrecordOutputsoptions are removed. Redact sensitive values before they reach the agent.
Tracing
- Error messages: Empty and missing failure text uses the same message in the TypeScript and Python SDKs. Serialized payloads use compact JSON.
Tool calls
- User-facing text: Set
userFacingMessagewhen a tool sends text directly to the person. Lemma shows that text as the message they received.
Mastra
- Agent name: The trace name comes from the agent entity, then the entity name, then the id inside the run span name. An explicit
agentNamestill wins.
Vercel AI
- Trace contract: One integration owns one in-flight run. The root records the current turn, thread, and user, with wall-clock times and structured output.
OpenAI Agents
- Trace contract: The processor records current-turn input and output, terminal errors, and span-bounded times. Soft tool errors match the Python SDK.
LangChain
- Trace contract: Owned runs keep current-turn input, output, and errors. Tool calls stay nested under one root until the final answer. LangGraph uses the LangChain adapter.
Mastra
- Exporter:
LemmaMastraExportermaps agent and workflow spans to Lemma traces, with nested generations and tools. Soft tool failures and the current user turn are included.
Tool calls
- MCP errors: A tool result with
isErroris recorded as an error, without an invented output. - Child timing: A tool span no longer ends after its parent generation.
Tracing
- Registry: An ended trace handle is released from the client registry.
Ingest
- One delivery: The
replaceoption is removed. Each execution sends one complete trace. Patching a trace over time is not supported.
Debug
- Ingest status:
LEMMA_DEBUG_VERIFYanddebugSmokeTestcheckGET /traces/ingest-statusfor up to 15 seconds, so a backlog does not look like a missed delivery. - Debug flag:
LEMMA_DEBUG=1is the preferred value.truestill works.
Debug
- Delivery logs: Debug mode logs client config and ingest responses, including failure hints.
- Smoke test:
debugSmokeTestchecks that a trace reached Lemma.
Ingest
- Send on end: Trace handles and the OpenAI Agents processor send a trace once, when it ends. They no longer send partial snapshots along the way.
Tracing
- First public release:
@uselemma/tracingsends traces to Lemma, with debug mode and experiment mode.
LangChain
- Callback handler: A LangChain callback handler records chains, model calls, and tools.
OpenAI Agents
- Processor: An OpenAI Agents processor records agent runs.
Vercel AI
- Telemetry: A Vercel AI SDK telemetry integration records model and tool calls.