LLM Agent Observability for Security Teams: LangSmith vs. Langfuse vs. Arize Phoenix

All three platforms trace LLM and agent calls, but they differ sharply on the two things a security buyer cares about most: where the trace data lives, and how long it stays there.

Pricing and free-tier limits change frequently. Confirm current numbers directly with each vendor before budgeting; treat the figures above as a starting point for a conversation, not a quote.

For an incident investigation, the question is not "does it trace," it's "does it trace enough to reconstruct the decision." All three capture the same broad categories of data: the full call tree from initial invocation through tool calls and sub-agent calls, the prompt and response at each step, latency, and (for RAG-backed agents) the retrieved context that was fed into the prompt. Where they diverge is structure and depth.

LangSmith renders the full agent execution tree natively for LangChain and LangGraph agents, with each node showing inputs, outputs, tool calls, and latency, plus tagging and metadata for filtering. Its Insights feature clusters failed runs by similarity, which is useful for triage but is pattern-matching, not a lifecycle audit trail; it will not automatically tell you that a decision at step 3 caused a failure at step 7.

Langfuse structures traces as nested observations (generations, spans, events) grouped into sessions, which maps well onto multi-turn agent conversations. It supports post-hoc scoring, so an analyst or an LLM-as-judge evaluator can attach a correctness or safety label to a trace after the fact, which is useful for building an audit record of review, but the scoring itself is an annotation layered on top of the trace, not part of the original capture.

Arize Phoenix and Arize AX both build on the OpenInference semantic convention over OpenTelemetry, capturing span attributes that support cohort analysis, drift detection, and comparisons across user segments, plus dedicated views for multi-agent conversation flows. This makes Arize's data model the most portable of the three for teams that already run an OpenTelemetry collector, since spans can be routed to other backends without a proprietary export step.

A point that matters more to a security buyer than to a developer: none of the three platforms markets itself as an immutable, tamper-evident audit log. All three allow traces to be deleted or, in some cases, edited via their own APIs by users with sufficient permissions. If a compliance requirement calls for write-once storage or cryptographic tamper evidence, plan to export traces out of whichever platform you choose and into a system built for that (a SIEM, an object store with immutability/legal-hold configured, or a dedicated audit log service). Our AI red teaming guide covers the adversarial testing side of this problem; observability tooling is the recording side, and the two are complementary, not substitutes.

For a security team, deployment model is often the deciding factor before feature comparison even starts, because it determines whether prompts, tool arguments, and retrieved documents (which may contain regulated or sensitive data) leave your network.

LangSmith is a managed SaaS product by default. LangChain also offers a bring-your-own-cloud (BYOC) option, generally available on AWS, and a fully self-hosted deployment where the control plane and Agent Servers run inside your own Kubernetes cluster so trace data never leaves your environment. Both of those options require the Enterprise plan. Teams with multi-region data residency requirements need a separate self-hosted instance and organization per region; there is no single-instance multi-geo mode.

Langfuse is open source under the MIT license and self-hosts on the same codebase and ClickHouse-based architecture as its managed cloud, via Docker Compose or Kubernetes, on every pricing tier including free. This is the most straightforward path for a team that wants to keep trace data entirely in-house without negotiating an enterprise contract first.

Arize splits into two products with different deployment stories. Arize Phoenix is free and self-hosted on SQLite or PostgreSQL, licensed under Elastic License 2.0, which is source-available rather than an OSI-approved open source license (it restricts offering Phoenix itself as a competing hosted service, but does not restrict internal use). Arize AX, the full commercial platform with production monitoring features, is SaaS by default, and self-hosting is available only to Enterprise customers.

The practical read: Langfuse and Arize Phoenix both let a data-sensitive team self-host without an enterprise sales conversation. LangSmith and Arize AX both gate self-hosting behind their top pricing tier.

LangSmith's integration is frictionless if your agents are built on LangChain or LangGraph: setting one environment variable turns on tracing with no code changes. That same tight coupling is a limitation for anything else. Instrumenting a custom agent framework or a non-LangChain stack means relying on OpenTelemetry instrumentation instead of the native integration, which works but loses some of the LangChain-specific tooling.

Langfuse and Arize (both Phoenix and AX) take the opposite approach: they are framework-agnostic from the start, with SDKs and OpenTelemetry/OpenInference support for LangChain, LlamaIndex, raw provider APIs (OpenAI, Anthropic, etc.), and custom agent code. If your environment is a mix of frameworks, or you expect to change frameworks later, this reduces the risk of re-instrumenting everything down the line.

For a security team specifically, OpenTelemetry support is the feature to weight most heavily regardless of vendor, because it is the mechanism that lets trace data flow into a downstream SIEM or log pipeline for retention and correlation with other security telemetry, rather than living only inside the observability vendor's own database.

LangSmith requires the least setup effort for a pure LangChain/LangGraph shop and the most effort for anything else. Its Enterprise self-hosted deployment (Kubernetes control plane plus Agent Servers) is a real infrastructure commitment, comparable to running any other stateful platform component in-house, and should be scoped with the same rigor as a SIEM or ticketing system migration.

Langfuse's self-hosted deployment is designed to be run by a small platform team: Docker Compose is enough for evaluation and small production workloads, and the same architecture scales to Kubernetes for larger deployments. Because it shares a codebase with the managed cloud offering, teams can start on cloud and move to self-hosted later (or the reverse) without a data model migration.

Arize Phoenix is the lightest of the three to stand up for a proof of concept, since SQLite requires no separate database server, but that same simplicity means production use at scale needs a deliberate move to PostgreSQL and its own capacity planning. Arize AX's operational burden sits with the vendor for SaaS customers and shifts to your team only if you negotiate Enterprise self-hosting.

Treat every number below as a snapshot; all three vendors adjust tiers and limits regularly, and none of the sources reviewed for this guide published a committed long-term price.

• LangSmith: a free tier around 5,000 traces per month, with paid plans introducing per-seat pricing (roughly $39 per seat per month at the Plus tier). Self-hosted and BYOC deployment require an Enterprise agreement, priced separately.
• Langfuse: a free tier with generous limits (self-hosted deployments are not metered by trace volume at all, since you run the infrastructure), and usage-based cloud pricing starting around $29 per month, metered by traces, observations, or scores rather than by seat.
• Arize AX: a free tier limited to short retention (15 days), a Pro tier around $50 per month with 30-day retention, and Enterprise pricing with custom retention, based on a hybrid of span count and ingestion volume in gigabytes. Arize Phoenix itself is free regardless of scale, since it is the self-hosted, source-available product.

For a security buyer, retention length deserves as much scrutiny as the dollar figure: a platform with a 15- or 30-day default retention window will not have the data an investigator needs if an incident is discovered weeks after it occurred, unless retention is explicitly upgraded or traces are exported to longer-term storage as they are generated.

Strengths: the fastest path to production tracing for teams already standardized on LangChain or LangGraph, a genuinely mature Enterprise self-hosted and BYOC option for teams that need data to stay on AWS, GCP, or Azure infrastructure they control, and a run-clustering feature (Insights) that speeds up triage of failure patterns across many traces.

Limitations: self-hosting and BYOC are Enterprise-only, so a team that needs on-premises deployment but isn't ready for an enterprise contract has no lighter-weight self-hosted option. Native tracing depth drops for non-LangChain frameworks, where you fall back to generic OpenTelemetry instrumentation. Multi-region data residency requires running and managing separate instances per region rather than one geo-aware deployment.

Best fit: a security team whose agents already run on LangChain/LangGraph, who can commit to (or already have) an Enterprise agreement, and whose data residency need is single-region or can be met by standing up one self-hosted instance per required region.

Strengths: MIT licensing and self-hosting available on every tier, including free, make it the most accessible option for a data-sensitive team that wants to keep every trace in-house from day one without a sales negotiation. It is framework-agnostic, supports post-hoc scoring and LLM-as-judge evaluation for building a review trail on top of captured traces, and offers up to three years of retention on its cloud Pro/Enterprise plans for teams that do use the managed service.

Limitations: session tracing does not automatically surface causal links between steps (it will not tell you that an early decision caused a later failure without manual review), and self-hosting, while genuinely open, is still infrastructure your team owns and must patch, back up, and monitor like any other stateful service.

Best fit: a security or platform team that needs self-hosted, data-sovereign trace storage now, without waiting on an enterprise procurement cycle, and that is comfortable running (or already runs) containerized infrastructure like Docker or Kubernetes.

Strengths: Phoenix is free, self-hosted, and requires no license negotiation for internal use, making it a low-friction way to get OpenTelemetry/OpenInference-based tracing running quickly, including for evaluation before committing to a paid platform. Arize AX adds production-grade monitoring on top: drift detection, alerting, cohort-based comparisons, and dedicated multi-agent conversation views that are useful for spotting anomalous agent behavior across many users, not just reconstructing one incident.

Limitations: Phoenix's Elastic License 2.0 is source-available, not OSI-approved open source, which matters if your organization has a policy requiring OSI-licensed tooling. Arize AX's default retention windows are short (15 days free, 30 days Pro), meaning a delayed incident discovery can outrun the data's lifespan unless retention is upgraded or traces are exported. Self-hosting the full AX platform is Enterprise-only, so most Arize customers below that tier are on SaaS.

Best fit: a team that wants OpenTelemetry-native tracing with strong drift and anomaly detection across a fleet of agents, is comfortable with Phoenix's licensing for self-hosted evaluation or lightweight production use, and either accepts SaaS for the full AX feature set or has the budget for Enterprise self-hosting.

Not every agent deployment justifies adopting a dedicated observability platform. If your agent is low-risk (it doesn't touch regulated data, doesn't take irreversible actions, and operates in a narrow, well-tested scope) structured application logging into your existing SIEM or log pipeline can capture the same core facts an investigator needs: the prompt, the tool calls with their arguments, the response, and a timestamp, all correlated with your existing incident response tooling instead of living in a separate vendor's UI.

This approach also avoids a real risk with all three platforms discussed above: introducing a new system that itself needs to be secured, access-controlled, and included in your own vendor risk assessment, potentially with sensitive prompts and retrieved documents flowing to a SaaS vendor you now have to vet. For a single internal tool with a handful of users, that overhead may not be justified.

Move from structured logging to a dedicated platform when any of the following becomes true: you're running multiple agents or agent frameworks and need a unified view, you need session-level or multi-agent trace correlation that flat logs don't give you cleanly, you need built-in evaluation or scoring workflows, or an auditor or regulator specifically asks for agent-level tracing evidence that your logging setup doesn't already produce in a reviewable form.

Run a fixed-length PoC (two to four weeks is typical) against your actual agent, not a demo notebook, and score each platform against these questions before committing: