BerriAI/litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

Python58K stars11K forks+301 today

View on GitHubShare on X

Topics

#anthropic#langchain#llm#llmops#openai#ai-gateway#azure-openai#bedrock#gateway#openai-proxy#vertex-ai#llm-gateway#litellm#mcp-gateway#rust#rust-ai

What it does

LiteLLM is a Python SDK and Proxy Server that allows developers to call over 100 large language models in OpenAI format, offering features like cost tracking, guardrails, load balancing, and logging. It simplifies the process of integrating various LLM APIs into applications.

Star history

2026-08-23 → 2026-09-06+1.1K stars

Tracking

Last trending: 2026-04-25

Creator kit

Hook

Simplify your AI integration with LiteLLM, a Python SDK and Proxy Server that supports over 100 LLM APIs in OpenAI format!

Content angles

Create tutorials on how to integrate different LLM providers using LiteLLM.
Develop case studies showcasing the cost-saving benefits of LiteLLM's load balancing feature.
Demonstrate the ease of setting up a proxy server for secure and efficient API calls with LiteLLM.

Who should care

Python developers, AI researchers, data scientists, and DevOps engineers working on projects involving large language models.