Self-hosted LLM observability tools in 2026
Trace prompts, model calls and agent steps, and track cost, latency and output quality of AI applications. This list has 20 LLM observability tools. Of these, 11 are open source, 20 have a free plan and 20 can be self-hosted. Checked by hand on 30 September 2026.
- 1
AgentOps
Observability for AI agents: records agent sessions with LLM calls, tool use and costs so you can replay and debug runs; SDK and app are open source.
- 2
Arize Phoenix
Source-available LLM tracing and evaluation tool built on OpenTelemetry and OpenInference; run it locally, in a notebook or on your servers to inspect traces and score outputs.
- 3
Braintrust
Evaluation and observability platform for AI products: logs production traces, runs evals and scorers, and offers a prompt playground and datasets.
- 4
Confident AI
Evaluation and observability platform from the makers of the open source DeepEval library: LLM tracing, regression tests in CI, and red teaming.
- 5
Future AGI
Open source platform to trace, evaluate, simulate and guard LLM and agent apps, including an AI gateway.
- 6
Galileo
AI evaluation and observability platform with tracing, evaluation metrics and real-time guardrails for agents; hosted, VPC or on-prem.
- 7
Helicone
Open source LLM proxy and logging: point your provider base URL at it to record every request with cost, latency and user, plus caching and rate limits.
- 8
HoneyHive
Observability and evaluation platform for AI agents: OpenTelemetry-based tracing, automated and human evals, prompt versioning and production monitoring.
- 9
Laminar
Open source observability for AI agents: OpenTelemetry-based tracing, evaluations and analysis of agent runs, self-hostable or cloud.
- 10
Langfuse
Open source LLM engineering platform: traces LLM and agent calls, manages prompts, runs evaluations and tracks cost and latency. Self-host or use the cloud; acquired by ClickHouse in 2026.
- 11
LangSmith
LangChain's platform for tracing, evaluating and monitoring LLM apps and agents, usable with or without LangChain; also hosts agent deployments.
- 12
LangWatch
Open source LLMOps platform: traces LLM and agent runs, runs evaluations and agent simulations, and helps optimize prompts; cloud or self-managed.
- 13
Latitude
Open source platform to trace, monitor and evaluate AI agents in production, with prompt management; self-host or use the hosted plans.
- 14
OpenLIT
Open source, OpenTelemetry-native AI observability: SDK auto-instruments LLMs, vector databases and GPUs, with a self-hosted UI for traces, costs, evals and prompts.
- 15
OpenLLMetry
Open source OpenTelemetry instrumentations for LLM providers, vector databases and agent frameworks, by Traceloop; sends traces to any OTLP backend or the Traceloop platform.
- 16
Opik
Open source LLM and agent observability from Comet: tracing, test suites, evaluations and a prompt optimizer; self-host the same code as the cloud.
- 17
Portkey
AI gateway with an observability layer: one API for many model providers with routing, fallbacks and caching, and logs of each request with cost and latency. The gateway is open source.
- 18
PostHog
Open source product suite with session replay, product analytics, feature flags, error tracking and LLM analytics sharing one event store. Hosted with a monthly free allowance, or self-hosted.
- 19
Respan
LLM gateway, logging and evaluation platform (formerly Keywords AI): routes calls to many models through one API and records traces, cost and scores.
- 20
W&B Weave
Weights & Biases toolkit for tracing LLM calls, running evaluations with custom scorers and comparing model and prompt versions.
Side by side
| Tool | Pricing | Free plan | Open source | Self-hosted | LLM call tracing | Agent and multi-step traces | Token and cost tracking | Prompt management | Evaluations | Datasets and experiments | User feedback | OpenTelemetry support | Playground | Alerts |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AgentOps | Free plan | Yes | Yes | Yes | Yes | Yes | Yes | - | - | - | - | Yes | No | - |
| Arize Phoenix | Free | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | - |
| Braintrust | Free plan | 1 GB processed data, 10k scores, 14-day retention | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Confident AI | Free plan | 2 seats, 1 project, 1 GB-month of trace spans | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | - | Yes |
| Future AGI | Free plan | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Galileo | Free plan | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Helicone | Free plan | 10,000 requests, 1 GB storage, 1 seat | Yes | Yes | Yes | Yes | Yes | Yes | - | Yes | Yes | Yes | Yes | Yes |
| HoneyHive | Free plan | 10,000 events/mo, 5 users, 30-day retention | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Laminar | Free plan | 1 GB data, 7-day retention, 1 seat | Yes | Yes | Yes | Yes | Yes | - | Yes | Yes | Yes | Yes | Yes | Yes |
| Langfuse | Free plan | 50k units/mo, 30 days data access, 2 users | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | - |
| LangSmith | Free plan | 5k base traces/mo, 1 seat | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| LangWatch | Free plan | 50k events/mo, 14-day data access, 2 users | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Latitude | Free plan | 20K credits/mo, 30-day retention | Yes | Yes | Yes | Yes | Yes | - | Yes | Yes | Yes | Yes | - | Yes |
| OpenLIT | Free | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | - | Yes | Yes | Yes | - |
| OpenLLMetry | Free plan | Traceloop: 50K spans/mo, 5 seats, 24h retention | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | - |
| Opik | Free plan | 25k spans/mo, 10 members, 60-day retention | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Portkey | Free plan | 10k logs/mo, 3-day log retention | No | Yes | Yes | Yes | Yes | Yes | - | - | Yes | Yes | Yes | Yes |
| PostHog | Free plan | 5K session recordings and 1M events per month | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Respan | Free plan | 100k logs, 1k scores | No | Yes | Yes | Yes | Yes | - | Yes | Yes | Yes | - | - | Yes |
| W&B Weave | Free plan | Yes | No | Yes | Yes | Yes | - | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
Yes: confirmed in the tool's documentation. No: documented as not offered. -: not confirmed. Plans and prices change; check each tool's own site before you decide.
More lists
- LLM observability tools in 2026 22
- Best LLM observability tools in 2026 14
- Open source LLM observability tools in 2026 11
- Self-hosted observability tools in 2026 96
- Self-hosted observability platforms in 2026 9
- Self-hosted metrics and time-series tools in 2026 9
- Self-hosted log management tools in 2026 17
- Self-hosted tracing and APM tools in 2026 11
- Self-hosted error tracking tools in 2026 7
- Self-hosted continuous profiling tools in 2026 9
- Self-hosted real user monitoring and session replay tools in 2026 6
- Self-hosted OpenTelemetry and telemetry pipeline tools in 2026 17
- Self-hosted dashboard tools in 2026 7