Appofficial publisher

LangSmith

App : A product you use directly

Observability and evaluation platform from LangChain, usable outside LangChain.

Natural choice if you are already on LangGraph; otherwise Langfuse is more open.

Official site

Overall score

8.1 /10

Security: nothing to flag

Utility

8.0

Reliability

6.0

Security

No caveat on record

Ease

5 min · Comfortable

Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.

What it does, concretely

When deploying an agent or a language model application, it is hard to know what is really happening inside: what decisions the agent makes, why a response is wrong, or where cost and latency spike. LangSmith addresses this need for visibility by recording and analysing every execution step.

In practice, LangSmith traces your agent's calls step by step, displays real-time monitoring dashboards covering cost, latency and error rates, and lets you run automated evaluations on response quality. It integrates through SDKs for Python, TypeScript, Go and Java, and supports OpenTelemetry to fit into existing observability infrastructure.

LangSmith is aimed at teams building agents or LLM pipelines in production who need to debug quickly, monitor performance and measure quality continuously. It works with any framework: OpenAI SDK, Anthropic SDK, LlamaIndex, Vercel AI SDK, or a custom implementation, not just LangChain.

However, if you are looking for a solution that is entirely neutral with respect to the LangChain ecosystem, or if your organisation prefers a more open community tool, alternatives such as Langfuse may be a better fit.

What it offers

·Step-by-step agent tracing with visualisation of every tool call and decision.
·Real-time monitoring dashboards covering latency (P50, P99), cost, error rates and feedback scores.
·Automated online evaluations (LLM-as-judge and code evals) to measure quality in production.
·Automatic trace clustering to detect usage patterns and recurring failure modes.
·OpenTelemetry support to integrate with existing observability infrastructure.
·Configurable alerts via webhooks or PagerDuty when a metric threshold is crossed.
·SmithDB, a database purpose-built for agent observability, with full-text search and sub-second JSON key-path filtering (paid).
·Managed cloud, BYOC or self-hosted deployment on a Kubernetes cluster for teams with data residency requirements (paid).

Our tips

Start with tracing only to validate the integration, then enable automated evaluations once you have identified the quality criteria that matter for your use case.
Use OpenTelemetry support if your team already has observability infrastructure: you can send LangSmith trace data to your existing tools or ingest OTel data into LangSmith.
Set up PagerDuty or webhook alerts on latency and cost metrics from the start, before production volumes make manual debugging impractical.
If your traces contain sensitive data, prefer the self-hosted or BYOC option so that data never leaves your environment.
Observability and evaluation are independent: you only pay for what you use, and there is no need to enable both modules at the same time.

Watch out

Cloud data is stored in GCP us-central-1 by default, which may raise compliance issues for organisations subject to data residency requirements outside the United States.
The tool is published by LangChain Inc., which may create ecosystem lock-in for teams that do not use LangGraph or LangChain.

In practice

Price: Freemium
Commercial use: Commercial use: yes
Available on: web

Security and data

Rating: Security: nothing to flag
Your data: Data with the vendor
Maintenance: Active

Links and repository checked on 25 October 2026 · added on 1 July 2023

How to install LangSmith

1. Create an account at smith.langchain.com and retrieve your API key from the settings. 2. Install the SDK for your language (for example: pip install langsmith for Python). 3. Set the LANGCHAIN_API_KEY environment variable with your API key; never commit it to your code repository. 4. Enable tracing by setting the environment variables LANGCHAIN_TRACING_V2=true and LANGCHAIN_PROJECT with your project name. 5. Run your application: traces appear automatically in the LangSmith dashboard. 6. If your data is sensitive, contact LangSmith about a BYOC or self-hosted option before going to production.

pip install langsmith

In the same category

Whole category Monitor →