LLM and AI observability
Trace prompts, model calls and agent steps, and track cost, latency and output quality of AI applications.
Lists: LLM observability tools in 2026 Best LLM observability tools in 2026 Open source LLM observability tools in 2026 Self-hosted LLM observability tools in 2026
-
Observability for AI agents: records agent sessions with LLM calls, tool use and costs so you can replay and debug runs; SDK and app are open source.
-
Arize's hosted AI observability and evaluation platform for agents and ML models: tracing, online evals, monitors and dashboards.
-
Source-available LLM tracing and evaluation tool built on OpenTelemetry and OpenInference; run it locally, in a notebook or on your servers to inspect traces and score outputs.
-
Evaluation and observability platform for AI products: logs production traces, runs evals and scorers, and offers a prompt playground and datasets.
-
Evaluation and observability platform from the makers of the open source DeepEval library: LLM tracing, regression tests in CI, and red teaming.
-
Open source platform to trace, evaluate, simulate and guard LLM and agent apps, including an AI gateway.
-
AI evaluation and observability platform with tracing, evaluation metrics and real-time guardrails for agents; hosted, VPC or on-prem.
-
Open source LLM proxy and logging: point your provider base URL at it to record every request with cost, latency and user, plus caching and rate limits.
Acquired by Mintlify 2026-03-03; service kept live in maintenance mode (security, bug fixes, new models). Source code
-
Observability and evaluation platform for AI agents: OpenTelemetry-based tracing, automated and human evals, prompt versioning and production monitoring.
-
Open source observability for AI agents: OpenTelemetry-based tracing, evaluations and analysis of agent runs, self-hostable or cloud.
-
Open source LLM engineering platform: traces LLM and agent calls, manages prompts, runs evaluations and tracks cost and latency. Self-host or use the cloud; acquired by ClickHouse in 2026.
-
LangChain's platform for tracing, evaluating and monitoring LLM apps and agents, usable with or without LangChain; also hosts agent deployments.
-
Open source LLMOps platform: traces LLM and agent runs, runs evaluations and agent simulations, and helps optimize prompts; cloud or self-managed.
-
Open source platform to trace, monitor and evaluate AI agents in production, with prompt management; self-host or use the hosted plans.
-
Open source, OpenTelemetry-native AI observability: SDK auto-instruments LLMs, vector databases and GPUs, with a self-hosted UI for traces, costs, evals and prompts.
-
Open source OpenTelemetry instrumentations for LLM providers, vector databases and agent frameworks, by Traceloop; sends traces to any OTLP backend or the Traceloop platform.
-
Open source LLM and agent observability from Comet: tracing, test suites, evaluations and a prompt optimizer; self-host the same code as the cloud.
-
AI gateway with an observability layer: one API for many model providers with routing, fallbacks and caching, and logs of each request with cost and latency. The gateway is open source.
-
Open source product suite with session replay, product analytics, feature flags, error tracking and LLM analytics sharing one event store. Hosted with a monthly free allowance, or self-hosted.
-
LLM gateway, logging and evaluation platform (formerly Keywords AI): routes calls to many models through one API and records traces, cost and scores.
-
Observability for AI and LLM apps on OpenTelemetry: traces model and tool calls with tokens, cost, latency and errors; send OTLP over HTTP or use the TypeScript SDKs.
-
Weights & Biases toolkit for tracing LLM calls, running evaluations with custom scorers and comparing model and prompt versions.
No tools match every filter. to see the full list.