LiteLLM

What it is

LiteLLM is an open-source LLM gateway from BerriAI, a Y Combinator company. It comes in two forms: a Python SDK that calls any model in the OpenAI request format, and a proxy server (the “AI Gateway”) that sits between your apps and your model providers. Every team calls one OpenAI-compatible endpoint; the proxy holds the real provider keys, issues virtual keys per team or agent, enforces budgets and rate limits, falls back between providers, and logs spend per call. The homepage claims 140+ providers and about 1,900 models; the docs still say 100+. The core is MIT-licensed. A paid Enterprise licence unlocks governance features on the same image.

The ops question it answers: five teams are calling OpenAI, Anthropic and Bedrock with keys pasted into n8n flows, scripts and Claude Code configs, finance cannot see who spent what, and revoking one leaked key means hunting through every tool.

The 2026 security record

This is the first thing a buyer should know, so it comes before features.

March 24, 2026. Versions 1.82.7 and 1.82.8 on PyPI carried a credential stealer for about 40 minutes before PyPI quarantined them. LiteLLM traced it to an unpinned Trivy scanner in CI that exposed static publishing credentials. The official proxy Docker image was not affected. Releases paused, Mandiant was brought in, and v1.83.0 shipped March 30 through a rebuilt pipeline with Trusted Publishing and cosign signing.

October 21, 2026. Version 1.107.0 was compromised again, this time on PyPI (about two hours) and on the Docker Hub and GHCR proxy images, including the latest and main-stable tags. The payload stole cloud credentials and every provider key stored in the proxy database, and installed a remote-code loader. LiteLLM’s October 23 root-cause post says an admin key leaked in a public repo in May was never revoked; the attacker used it to reach an org owner’s GitHub token and pointed the real, signed release pipeline at a lookalike fork. Trusted Publishing and signing worked as designed and did not help, because the attacker held owner rights. Enterprise customers on the private registry were not affected. The safe pin was 1.106.2; clean releases signed with a rotated cosign key followed from October 24.

The October 30 follow-up lists dated fixes: a single hardware-bound release approver and Splunk-streamed audit logs are done; phishing-resistant MFA and secret scanning by November 30; an out-of-band verifier that auto-yanks mismatched artifacts by December 31. The project is alive and shipping (1.108.4 landed November 9), but those are plans, not a track record.

What you are actually buying

Open source, free. The SDK and proxy with virtual keys, per-key budgets and RPM/TPM limits, spend tracking, the Admin UI, load balancing and fallbacks, the guardrail framework, Presidio PII masking, an MCP gateway, and logging callbacks to Langfuse, LangSmith, Arize Phoenix and OpenTelemetry. SSO for the Admin UI is free up to 5 users.

Enterprise licence. A LITELLM_LICENSE key on the same image, plus access to the private release registry. It adds SSO beyond 5 users, SCIM, JWT auth, audit logs with retention, RBAC across orgs and teams, secret-manager backends (AWS, Azure, Google, HashiCorp Vault, CyberArk), IP allowlists, key rotation, tag-based and per-model budgets, per-team logging controls, licensed guardrail integrations such as Lakera prompt-injection checks, and support SLAs (Sev 0 in 1 hour; security patches in 72 hours).

Pricing

Checked 2026-11-10 at litellm.ai and docs.litellm.ai. Open source is $0. Enterprise has no published price: the docs say pricing “depends on your deployment size” and is quote-only, sold direct, by invoice, or through AWS and Azure Marketplace, with a self-serve 30-day trial key. LiteLLM itself aims Enterprise at teams with 100+ users or 10+ production AI use cases. Third-party guides still repeat a roughly $30,000/year figure from the 2024 launch; treat it as an unverified anchor for budgeting, not a quote.

The real cost of the free tier is people: a Postgres database, Redis for multi-instance rate limits, and an engineer who owns upgrades and incident response. Budget a fraction of one platform engineer before you call it free.

Best for

A platform or RevOps engineering team that already runs containers, needs one place to issue, cap and revoke model keys across several providers, and wants the gateway inside its own VPC for data-residency reasons. For a company that has made it a production dependency, Enterprise is now the sensible tier, mainly for the private registry that dodged the October compromise.

Not for

A team without someone on call for infrastructure; a managed gateway costs less than one bad upgrade. Not for a single-provider shop: if every call goes to Bedrock or Azure, their native quotas and IAM already do most of this. And not as your observability layer: it logs spend, but traces and evals belong in Langfuse or LangSmith.

Versus the alternatives

• Amazon Bedrock or Microsoft Foundry — pick the cloud’s own model platform when 80%+ of calls already go through one cloud and IAM is your access control.
• OpenRouter — the fastest-growing hosted option, now owned by Stripe. Pick it when you want one bill across hundreds of models and no infrastructure, and you accept a third party in the request path.
• Portkey — open-source gateway with a hosted control plane, acquired by Palo Alto Networks in May 2026 and folded into Prisma AIRS. Pick it when your security team already buys from Palo Alto.
• Kong AI Gateway or Cloudflare AI Gateway — pick these when that vendor already fronts your APIs and you want model traffic under the same policies.
• For agent-to-tool traffic rather than model traffic, Runlayer governs MCP servers; LiteLLM’s MCP gateway is lighter.

So is self-hosted LiteLLM still safe? The code is not the problem; the release channel was, twice in seven months. Keep it if you can pin versions, verify signatures and contain blast radius. Otherwise, pay for Enterprise or move to a managed gateway.

Watch-outs

• Floating tags pulled the October malware. Anyone deploying latest or main-stable got the bad image within minutes. Guard: pin exact versions by image digest, verify cosign signatures against the rotated key, and wait 48 hours after a release before promoting it to production.
• The proxy database is a vault of every provider key. That is exactly what the October payload stole. Guard: store provider keys in a secret manager rather than the proxy DB (an Enterprise feature), give the proxy host least-privilege IAM with no broad cloud role, and restrict its outbound network to your providers’ endpoints.
• Rotation after an incident is large. Remediation meant rotating the host’s IAM role, every stored provider key, the master key, all virtual keys and the database password. Guard: script and rehearse key rotation once a quarter so an incident takes an hour, not a week.
• Free SSO stops at 5 users. Past that, Admin UI access needs the licence. Guard: limit Admin UI access to the platform team and hand everyone else virtual keys, or budget Enterprise before the sixth admin needs in.