Phoenix

About Phoenix

Arize Phoenix is a source-available observability and evaluation tool for LLM and ML systems.

Best for: Teams who need a source-available LLM observability platform with a built-in web UI for trace inspection, RAG retrieval evaluation, embedding visualization, and experiment comparison — running locally without requiring a cloud account.

Deployment: Local Python server launched via pip install arize-phoenix and a single Python call; also available as Arize Phoenix Cloud and self-hosted Docker deployment

Skill level: Intermediate

Tradeoffs

Local server mode does not persist traces across restarts by default — production deployments require configuring external storage or using Arize Phoenix Cloud. Integration requires adding OpenTelemetry or Phoenix SDK instrumentation to application code. LLM-as-judge scoring adds API latency and cost when running automated evaluations at scale across large trace datasets. Licensing is the constraint to check first: Phoenix ships under the Elastic License 2.0, which is source-available rather than OSI open source. Internal self-hosting is fine, but you may not offer Phoenix to third parties as a hosted or managed service, and you may not circumvent its license-key functionality — so if your plan is to resell observability to your own customers, you need a separate agreement with Arize.

Related guides and resources

Explore step-by-step setup guides, comparisons, and stack recipes for this tool category.

Best for

Teams who need a source-available LLM observability platform with a built-in web UI for trace inspection, RAG retrieval evaluation, embedding visualization, and experiment comparison — running locally without requiring a cloud account.

Why use it

Phoenix runs locally as a lightweight Python server, making it the fastest path to LLM tracing and RAG evaluation without cloud dependency. Its retrieval evaluation UI shows exactly which documents were retrieved for each query, how they scored against the RAG triad (context relevance, faithfulness, answer relevance), and how the model used them — giving engineers debugging visibility that log files cannot provide.

Key features

• OpenTelemetry-compatible trace collection with a built-in web UI for inspecting LLM spans, latencies, and token counts without cloud setup
• RAG evaluation using the RAG triad metrics: context relevance, response faithfulness, and answer relevance via LLM-as-judge
• Embedding visualization for clustering prompt embeddings and detecting distribution shifts in production input populations
• Experiment comparison UI for A/B testing prompt changes, retrieval configurations, and model versions on evaluation datasets

Tradeoffs

Local server mode does not persist traces across restarts by default — production deployments require configuring external storage or using Arize Phoenix Cloud. Integration requires adding OpenTelemetry or Phoenix SDK instrumentation to application code. LLM-as-judge scoring adds API latency and cost when running automated evaluations at scale across large trace datasets. Licensing is the constraint to check first: Phoenix ships under the Elastic License 2.0, which is source-available rather than OSI open source. Internal self-hosting is fine, but you may not offer Phoenix to third parties as a hosted or managed service, and you may not circumvent its license-key functionality — so if your plan is to resell observability to your own customers, you need a separate agreement with Arize.

Alternatives

• Langfuse
• Ragas
• DeepEval