Appofficial publisher

LiteLLM

App : A product you use directly

Self-hosted gateway that unifies 100+ providers behind an OpenAI API, with budgets and virtual keys.

The workhorse of self-hosted gateways; pin versions after the March 2026 incident.

Official site GitHub · 61,985 ★

Overall score

7.8 /10

Security: one caveat, see the card

Utility

9.0

Reliability

10.0

Security

One caveat

Ease

Technical · Technical

Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.

What it does, concretely

Managing calls to multiple language model providers usually means juggling different SDKs, authentication patterns, and incompatible request formats. LiteLLM solves this by exposing a single OpenAI-compatible API that covers 100+ providers, from OpenAI and Anthropic to AWS Bedrock, Azure, Gemini, and many more.

In practice, LiteLLM is deployed as a proxy server (via Docker, Helm, or Terraform) in front of your applications. Each request reaches the gateway, which translates it into the target provider's format, applies budget and rate-limit rules, and returns the response in a unified format. An admin dashboard lets you track spend by team, virtual key, or model, and cut off access as soon as a cap is reached.

LiteLLM is aimed at engineering teams and platform teams that want to centralize model access across an entire organization: developers who want to switch providers without rewriting code, managers who need to trace and cap spending, or teams operating in air-gapped environments. The gateway also supports agents via the A2A protocol and MCP servers.

On the other hand, LiteLLM is not suited for general consumers or anyone looking for a graphical interface to chat with a model. Installation and configuration require system administration or DevOps skills. Advanced governance, chargeback, and auto-routing features are reserved for the paid enterprise plan.

What it offers

·Unified OpenAI-compatible API for 100+ providers and 1,800+ models, with no code changes required when switching providers.
·Virtual keys with budgets and rate limits per key, team, organization, or model, with automatic request blocking when a cap is reached.
·Admin dashboard to track spend, requests, and tokens in real time by team, key, or model.
·Load balancing and auto-routing to the cheapest or most capable deployment based on request complexity (paid).
·Response caching (Redis, S3, GCS) and prompt compression to reduce costs and latency.
·Configurable guardrails: PII detection, prompt injection, secret detection, toxicity filtering, via integrations with Presidio, Lakera, Aporia, and others.
·MCP gateway to connect MCP servers to any model through the same API, and A2A protocol support for agents.
·Enterprise chargeback: attribute every request to a team or business unit for internal billing (paid).

Our tips

Pin a specific Docker image version rather than using the latest tag: an incident in March 2026 showed that automatic updates can introduce regressions in production.
Set strict monthly budgets per team on virtual keys from the start: it is easier to raise a cap than to deal with an unexpected bill after an overage.
Enable response caching with Redis for repetitive requests (evaluations, tests) to avoid paying twice for the same answer.
For air-gapped environments, LiteLLM can be deployed without outbound internet access by pointing only to self-hosted models or internal endpoints.
Configure PII-detection guardrails at the pre-call stage if your users send sensitive data: the gateway sees every prompt and can mask information before it reaches the provider.

Watch out

The gateway sees every prompt and stores provider API keys: misconfigured access controls or a compromised server exposes both data and provider credentials.
Advanced features (auto-routing, chargeback, SSO) are reserved for the paid enterprise plan whose pricing is not publicly listed, making cost comparison difficult before committing.

In practice

Price: Freemium
Commercial use: Commercial use: yes
Available on: docker, linux, cli
Licence: MIT

Security and data

Rating: Security: one caveat, see the card
Your data: Data stays local
Known incidents: 24 mars 2026 : versions PyPI 1.82.7 et 1.82.8 malveillantes publiées via des identifiants volés (compromission CI/CD Trivy), retirées en quelques heures.
Maintenance: Active
Last release: 24 October 2026

Links and repository checked on 25 October 2026 · added on 1 August 2023

How to install LiteLLM

1. Pull the official Docker image: docker pull docker.litellm.ai/berriai/litellm:latest (prefer a pinned version in production). 2. Create a config.yaml file listing your authorized providers and models. 3. Store provider API keys in environment variables or your existing secrets manager, never in plain text in the configuration file. 4. Start the container by mounting the configuration file: docker run -p 4000:4000 -v $(pwd)/config.yaml:/app/config.yaml litellm. 5. Verify the gateway responds at http://localhost:4000 and create your first virtual keys from the admin dashboard. 6. Set budgets and rate limits per team before opening access to your users.

docker pull docker.litellm.ai/berriai/litellm:latest && docker run -p 4000:4000 litellm

In the same category

Whole category Route & save →