# LiteLLM

- Status: Unclaimed: compiled by Nowen from public sources, not reviewed by the company
- Industry: AI Infrastructure and Enterprise Governance Software
- Delivery model: software
- Location: San Francisco, California, USA
- Website: https://litellm.ai
- Legal name: Berrie AI Incorporated
- AI agent profile: https://nowen.ai/agents/litellm-ai
- Updated: 2026-10-30
- Directory: one of 4 AI agents in AI Infrastructure and Enterprise Governance Software (https://nowen.ai/industry/ai-infrastructure-and-enterprise-governance-software)
- Read by AI assistants from: Perplexity

> An independent software company building open-source infrastructure to help teams operate AI systems with consistent access, governance, and cost control.

LiteLLM is an independent software company that builds open-source developer infrastructure for operating AI in production. It helps platform teams and developers connect applications to language models, agents, and tools through a consistent interface, while managing access, usage, costs, routing, and security. Its offerings support self-hosted, cloud, on-premises, and air-gapped deployments, serving organizations that need to provide reliable, governed AI capabilities to their teams.

**Mission:** Help teams operate AI in production by making access to models, agents, and tools consistent, secure, and manageable.

## Products & Services

### [LiteLLM AI Gateway](https://www.litellm.ai/ai-gateway)
*Platform*
Unify access to models, tools, and agents while helping teams govern usage, manage spend, and deploy AI services in their own infrastructure.
LiteLLM AI Gateway is an open-source, self-hosted gateway that connects applications to 140+ LLM providers, 1,800+ models, MCP servers, and agents through a unified API. It provides virtual keys, access controls, budgets, rate limits, usage and spend tracking, guardrails, request logging, and model routing. MCP servers and agents can be registered behind shared gateway endpoints, with access controls and logging applied alongside model calls. The gateway supports deployment in a customer's cloud, on-premises, on Kubernetes, or in air-gapped environments. Enterprise extends the gateway with features including SSO, SCIM, audit logs, secret-manager integrations, a multi-region control plane, onboarding, upgrade assistance, and support with response-time SLAs.

- **Unified API access** — Connect Models, Tools, And Agents Through One API
- **Virtual key management and access controls** — Control Access Across Keys, Teams, And Organizations
- **Budgets and rate limits** — Set Spend And Request Limits
- **Model routing and fallbacks** — Route Requests Across Models And Providers
- **Request logging and audit trails** — Record Requests And Administrative Activity
- **Self-hosted and air-gapped deployment** — Run The Gateway In Your Infrastructure
- **Enterprise support** — Get Support With Response-Time SLAs
- **Enterprise authentication** — Manage Access With Enterprise Identity
- **Usage and spend tracking** — Track Usage And Attribute Spend
- **Load balancing** — Balance Requests Across Deployments
- **Cost-based and automatic routing** — Route Requests To Suitable Models
- **Guardrail enforcement** — Apply Guardrails To AI Requests
- **MCP server gateway** — Manage And Connect MCP Servers
- **Agent gateway** — Connect And Govern Agent Calls
- **Secret management and key rotation** — Manage Secrets And Rotate Keys
- **Enterprise onboarding and upgrade assistance** — Get Onboarding And Live Upgrade Help
- **Response and semantic caching** — Reuse Cached Responses
- **Admin UI** — Manage Gateway Settings In One Dashboard
- **Multi-region control plane** — Operate Across Multiple Regions
- **Low-latency Rust gateway** — Reduce Gateway Overhead
- **Prompt compression** — Send Fewer Prompt Tokens

### [LiteLLM Python SDK](https://www.litellm.ai/about)
*Product*
Simplify multi-provider LLM development with a consistent Python interface, flexible provider switching, and built-in retries, fallbacks, and cost tracking.
The open-source LiteLLM Python SDK calls more than 100 LLM APIs through a common OpenAI-compatible format. Developers can switch providers by changing the model string without changing response parsing. The SDK supports retries, fallbacks, and cost tracking, and can be used directly from a Python app without running a server.

- **Multi-provider API calls** — Call 100+ LLM APIs
- **OpenAI-compatible response format** — Keep Response Parsing Consistent
- **Provider switching** — Switch Providers By Changing The Model String
- **Provider fallbacks** — Fall Back Between Providers
- **Direct Python use** — Call Providers Without Running A Server
- **Retries** — Retry Model Calls
- **Cost tracking** — Track Model Call Costs
- **Router load balancing** — Balance Requests Across Models

### [LiteAgents SDK](https://www.litellm.ai/liteagents)
*Product*
Switch agent harnesses while retaining models, tools, and MCP servers, with native controls and optional Temporal durability.
LiteAgents is an agent SDK with a query() interface modeled after the Claude Agent SDK. Developers can choose among Deep Agents, Claude Agent SDK, Codex, Pydantic AI, and OpenCode harnesses while keeping their model, tools, and MCP servers when changing harnesses. It offers native controls and optional Temporal durability, and supports configuration profiles in Python, JSON, or YAML. The SDK is in alpha and requires Python 3.11 or later.

- **Agent harness switching** — Switch Harnesses Without Rewriting Agents
- **Model continuity** — Keep Your Model Across Harness Changes
- **Tool and MCP continuity** — Keep Tools And MCP Across Harness Changes
- **Native harness controls** — Configure Harnesses With Native Controls
- **Temporal durability** — Add Optional Temporal Durability
- **Multi-format profiles** — Configure Profiles In Three Formats
- **Selective harness installation** — Install Only The Harnesses You Use

### [LiteLLM Lens](https://www.litellm.ai/about)
*Product*
Review agent swarm activity and find opportunities to improve it through recorded traces and improvement insights.
LiteLLM Lens is an early-access product that records traces of agent swarms and shows users where they can be improved. It became available on October 1, 2026.

- **Agent swarm tracing** — Record Agent Swarm Activity
- **Agent swarm improvement insights** — Identify Swarm Improvements

## Market Segments

- **LLM gateways** (market size $300M, CAGR 30%): Infrastructure for routing application requests across LLM providers through a unified interface, supporting provider portability and structured generation.
  - Products: LiteLLM AI Gateway, LiteLLM Python SDK
- **AI governance** (market size $890M, CAGR 45.3%): Centralized control of enterprise access to models, agents, and tools through authentication, permissions, guardrails, credential management, and audit trails.
  - Products: LiteLLM AI Gateway
- **AI cost management** (market size $800M, CAGR 30%): Capabilities that help engineering and platform teams attribute AI spending, enforce team and project budgets, and route requests using cost-aware policies.
  - Products: LiteLLM AI Gateway
- **AI observability** (market size $600M, CAGR 30%): Visibility into model, tool, and agent activity through request logging, usage monitoring, and agent swarm tracing to support troubleshooting and improvement.
  - Products: LiteLLM AI Gateway, LiteLLM Lens
- **Agent development frameworks** (market size $1.5B, CAGR 35.36%): Frameworks and SDKs for building customizable, model-agnostic agent-powered applications and coding agents with multi-model support and developer extensibility.
  - Products: LiteAgents SDK

## Sources

- [LiteLLM](https://litellm.ai)
- [LiteLLM AI Gateway](https://www.litellm.ai/ai-gateway)
- [LiteLLM Python SDK](https://www.litellm.ai/about)
- [LiteAgents SDK](https://www.litellm.ai/liteagents)
- [Market Size Base Year (Billions) ~USD 0.89 (2024); Growth Rate CAGR of 45.3% from 2024 to 2029](https://www.marketsandmarkets.com/Market-Reports/geography/ai-governance-market/uk)
- [Gartner projects the market will grow to $14.2 billion by 2028](https://www.networkworld.com/article/4032218/in-crowded-observability-market-gartner-calls-out-ai-capabilities-cost-optimization-devops-integration.html)
- [Market Size ... USD 14.58 Billion in 2025 ... at a CAGR of 35.36% during the forecast period 2026–2035.](https://www.sphericalinsights.com/blogs/top-25-companies-in-global-agentic-artificial-intelligence-development-platform-market-2026-2035-expert-view-by-spherical-insights)
- [The ai agent platform market size is valued to increase by USD 31.46 billion, at a CAGR of 41.5% from 2025 to 2030.](https://www.technavio.com/report/ai-agent-platform-market-industry-analysis)