What is Lemma?
Lemma provides:- Distributed Tracing — Track your agent’s execution flow with OpenTelemetry-based tracing
- Metric Events — Capture feedback signals and quality indicators linked to specific traces
- Experiments — Systematically evaluate different strategies against consistent test cases
- Analytics — Filter, aggregate, and analyze your agent’s performance over time
Key Features
OpenTelemetry-Based Tracing
Lemma uses OpenTelemetry as its foundation for distributed tracing. This means:- Standard, vendor-neutral instrumentation
- Rich ecosystem of libraries and integrations
- Detailed visibility into nested operations and timing
- Automatic capture of model calls, token usage, and tool invocations
Multiple Framework Support
Lemma integrates seamlessly with popular AI frameworks:- Vercel AI SDK — Native telemetry support with
streamTextandgenerateText - Langfuse — Automatic span capture via OpenTelemetry
- Claude Agent SDK — Direct integration for Claude-based agents
- LangGraph — Trace complex agent workflows and state machines
Flexible Metric Collection
Connect feedback signals to specific agent executions:- User feedback (thumbs up/down, ratings, flags)
- Automated quality checks (moderation, validation)
- Downstream outcomes (conversions, resolutions)
Systematic Experimentation
Compare different strategies with controlled experiments:- Define test cases once, run multiple strategies
- Side-by-side comparison in the dashboard
- Track metrics across strategy variations
- Identify patterns and edge cases
Getting Started
- Set up tracing — Choose your framework integration and send traces to Lemma
- Record metrics — Capture feedback and quality signals
- Run experiments — Systematically evaluate your agent’s performance