LiteLLMUnclaimed

LiteLLM is an open-source AI gateway that sits between your applications and the model providers you use. Applications call one OpenAI-compatible API, and the gateway forwards each request to providers such as OpenAI, Anthropic, Google Vertex AI and AWS Bedrock. The page says it covers 140+ providers and 1,800+ models, and that agents and MCP servers can be reached through the same gateway.

The product is aimed at platform teams who need to give a whole company access to models without handing out raw provider keys. Keys, teams and single sign-on are managed in one place, and access can be scoped by team, project or app. The page says it works with the secret manager a company already runs.

Spend is tracked per key, user, team, organisation, tool, agent and MCP server. Budgets and rate limits can be set per key, team, organisation and model, with daily or monthly resets. When a budget is reached, requests stop, or the gateway can throttle or fall back to a cheaper model. The page also describes guardrails for PII masking, prompt injection and secret detection, along with an audit log for each request.

Routing is handled by load balancing across providers, regions and keys, by lowest-cost routing, and by an Auto Routing feature that sends simple prompts to cheaper models and harder ones to stronger models. Response and semantic caching is supported through Redis, S3 and GCS, and prompt compression is listed as a way to send fewer tokens.

LiteLLM can be self-hosted on Linux, Docker, Kubernetes or macOS, including in an air-gapped environment. The page includes a quick-start install script and a Python SDK. The open-source edition is listed as MIT-licensed and free to self-host. The page says an Enterprise tier adds SSO, RBAC, audit logs, SCIM and support, and it offers a sales contact for that tier without stating a price.