LiteLLM

Unify access to models, tools, and agents while helping teams govern usage, manage spend, and deploy AI services in their own infrastructure.

Simplify multi-provider LLM development with a consistent Python interface, flexible provider switching, and built-in retries, fallbacks, and cost tracking.

Switch agent harnesses while retaining models, tools, and MCP servers, with native controls and optional Temporal durability.

Review agent swarm activity and find opportunities to improve it through recorded traces and improvement insights.

Infrastructure for routing application requests across LLM providers through a unified interface, supporting provider portability and structured generation.

Products: LiteLLM AI Gateway, LiteLLM Python SDK

Centralized control of enterprise access to models, agents, and tools through authentication, permissions, guardrails, credential management, and audit trails.

Products: LiteLLM AI Gateway

Capabilities that help engineering and platform teams attribute AI spending, enforce team and project budgets, and route requests using cost-aware policies.

Products: LiteLLM AI Gateway

Visibility into model, tool, and agent activity through request logging, usage monitoring, and agent swarm tracing to support troubleshooting and improvement.

Products: LiteLLM AI Gateway, LiteLLM Lens

Frameworks and SDKs for building customizable, model-agnostic agent-powered applications and coding agents with multi-model support and developer extensibility.

Products: LiteAgents SDK

LiteLLM AI Gateway

LiteLLM AI Gateway is an open-source, self-hosted gateway that connects applications to 140+ LLM providers, 1,800+ models, MCP servers, and agents through a unified API. It provides virtual keys, access controls, budgets, rate limits, usage and spend tracking, guardrails, request logging, and model routing. MCP servers and agents can be registered behind shared gateway endpoints, with access controls and logging applied alongside model calls. The gateway supports deployment in a customer's cloud, on-premises, on Kubernetes, or in air-gapped environments. Enterprise extends the gateway with features including SSO, SCIM, audit logs, secret-manager integrations, a multi-region control plane, onboarding, upgrade assistance, and support with response-time SLAs.

• Connect Models, Tools, And Agents Through One API
  Give applications a consistent gateway interface for models, MCP tools, and agents, and change model configurations without changing application code.
• Control Access Across Keys, Teams, And Organizations
  Provide scoped access for applications, users, teams, and organizations. Agent and MCP calls can use the same keys and access controls as model calls.
• Set Spend And Request Limits
  Apply usage limits at the levels identified by the organization and stop requests when budget caps are reached.
• Route Requests Across Models And Providers
  Direct traffic to configured models and providers, and use fallbacks to handle provider outages.
• Record Requests And Administrative Activity
  Review request activity and administrative changes, including agent and tool calls, and send logs to supported observability systems.
• Run The Gateway In Your Infrastructure
  Deploy the gateway in cloud, on-premises, Kubernetes, or air-gapped environments, keeping AI traffic and provider keys within your infrastructure.
• Get Support With Response-Time SLAs
  Access support with response targets for production incidents and security patches. Custom SLAs are also available.
• Manage Access With Enterprise Identity
  Connect enterprise identity and provisioning workflows through SSO and SCIM, and use OIDC or JWT authentication for gateway access.
• Track Usage And Attribute Spend
  See which users, teams, tools, agents, and services drive activity and costs, with attribution to support chargeback and organizational oversight.
• Balance Requests Across Deployments
  Distribute requests across configured models, providers, regions, and keys.
• Route Requests To Suitable Models
  Choose a lower-cost deployment when it can serve the request, or route prompts to models according to their complexity.
• Apply Guardrails To AI Requests
  Configure checks and policies for requests, including controls for sensitive information, prompt injection, and secrets.
• Manage And Connect MCP Servers
  Make registered MCP servers available through shared endpoints, control which keys, teams, or organizations can access them, and connect servers using supported transports. OpenAPI specifications can be converted into MCP servers, and tools can also be called through a REST endpoint without an LLM.
• Connect And Govern Agent Calls
  Route supported agent calls through the gateway and manage their access, logging, usage, and spend alongside model calls.
• Manage Secrets And Rotate Keys
  Use secret-manager integrations and key rotation to manage gateway credentials.
• Get Onboarding And Live Upgrade Help
  Receive dedicated onboarding and help from the LiteLLM team during live upgrades.
• Reuse Cached Responses
  Use response and semantic caching to avoid paying repeatedly for the same answer.
• Manage Gateway Settings In One Dashboard
  Give teams a dashboard for managing keys, models, budgets, MCP servers, and cost data.
• Operate Across Multiple Regions
  Use a multi-region control plane to support enterprise gateway operations across regions.
• Reduce Gateway Overhead
  The published benchmark reports 0.66 ms of p99 added latency for the Rust gateway under the stated test conditions.
• Send Fewer Prompt Tokens
  Reduce the number of tokens sent in prompts.

LiteLLM Python SDK

The open-source LiteLLM Python SDK calls more than 100 LLM APIs through a common OpenAI-compatible format. Developers can switch providers by changing the model string without changing response parsing. The SDK supports retries, fallbacks, and cost tracking, and can be used directly from a Python app without running a server.

• Call 100+ LLM APIs
  Use one Python SDK to call more than 100 LLM APIs from Python applications.
• Keep Response Parsing Consistent
  Handle provider responses in a common format so switching providers does not require changing response-parsing code.
• Switch Providers By Changing The Model String
  Change the model string to use a different provider while keeping the application's response handling consistent.
• Fall Back Between Providers
  Use provider fallbacks to keep model calls from depending on a single provider.
• Call Providers Without Running A Server
  Use the SDK directly in a Python application, without deploying a separate gateway server.
• Retry Model Calls
  Use SDK-supported retries when making model calls.
• Track Model Call Costs
  Track costs associated with model calls made through the SDK.
• Balance Requests Across Models
  Use the Router to load balance requests across models or providers from a Python application.

LiteAgents SDK

LiteAgents is an agent SDK with a query() interface modeled after the Claude Agent SDK. Developers can choose among Deep Agents, Claude Agent SDK, Codex, Pydantic AI, and OpenCode harnesses while keeping their model, tools, and MCP servers when changing harnesses. It offers native controls and optional Temporal durability, and supports configuration profiles in Python, JSON, or YAML. The SDK is in alpha and requires Python 3.11 or later.

• Switch Harnesses Without Rewriting Agents
  Select a different supported harness while using the same query() interface, avoiding the need to rewrite the agent when changing harnesses.
• Keep Your Model Across Harness Changes
  Change the selected harness without replacing the model configured for the agent.
• Keep Tools And MCP Across Harness Changes
  Retain the agent's tools and MCP servers when switching between supported harnesses.
• Configure Harnesses With Native Controls
  Apply native configuration options for the selected harness through LiteAgents profiles.
• Add Optional Temporal Durability
  Enable optional Temporal durability for agent workflows.
• Configure Profiles In Three Formats
  Choose Python, JSON, or YAML to configure LiteAgents profiles.
• Install Only The Harnesses You Use
  Limit installation to the harness integrations needed for your project; the SDK supports Python 3.11 and later.

LiteLLM Lens

LiteLLM Lens is an early-access product that records traces of agent swarms and shows users where they can be improved. It became available on October 1, 2026.

• Record Agent Swarm Activity
  Capture traces of agent swarms so users can review their activity.
• Identify Swarm Improvements
  Surface areas where agent swarms can be improved, helping users understand where to focus.