Migrate from LiteLLM

LiteLLM works until traffic spikes, a provider goes down, or someone has to redeploy at 2am. Then it stops being a library and starts being a job. Mesh API is the managed version of what your LiteLLM proxy is trying to be: one OpenAI-compatible endpoint, 1,000+ models, automatic failover, real-time spend visibility, and the same SDK you already use.

Quick Migration

Both services speak the OpenAI-compatible format. Migration is a two-line change.

- base_url = "http://localhost:4000/v1"     # LiteLLM proxy+ base_url = "https://api.meshapi.ai/v1"- api_key  = os.environ["LITELLM_API_KEY"]+ api_key  = os.environ["MESH_API_KEY"]

That's the migration. Everything else | model names, request shape, streaming, tool calls | stays the same.

Why teams switch from LiteLLM

Migration steps

1. Get a Mesh API key

Sign up at meshapi.ai, hit the dashboard, create a key. ~30 seconds.

2. Decide how you want to route

Mesh supports two modes:

• model="auto" | Mesh picks the best model for the request, live-scored on cost, latency, capability, and context window. This is the 40% saver.
• model="anthropic/claude-opus-4-7" or any specific ID | You pin a model, Mesh handles failover if that provider has trouble.

Most teams start by leaving their existing model names in place (they all map directly) and graduate to "auto" for non-critical paths.

3. Swap base URL and key

Python (OpenAI SDK):

from openai import OpenAI # Before | LiteLLM proxyclient = OpenAI(    base_url="http://localhost:4000/v1",    api_key=os.environ["LITELLM_API_KEY"],) # After | Mesh API (same SDK, same model name)client = OpenAI(    base_url="https://api.meshapi.ai/v1",    api_key=os.environ["MESH_API_KEY"],) resp = client.chat.completions.create(    model="auto",   # or keep "gpt-4.1", "claude-opus-4-7", etc.    messages=[{"role": "user", "content": "Hello!"}],)

Python (LiteLLM library, if you still want to use it):

import litellm, os resp = litellm.completion(    model="gpt-4.1",    messages=[{"role": "user", "content": "Hello!"}],    api_base="https://api.meshapi.ai/v1",    api_key=os.environ["MESH_API_KEY"],)

TypeScript / JavaScript:

import OpenAI from "openai"; const client = new OpenAI({  baseURL: "https://api.meshapi.ai/v1",  apiKey: process.env.MESH_API_KEY,}); const completion = await client.chat.completions.create({  model: "auto",  messages: [{ role: "user", content: "Hello!" }],});

cURL:

curl https://api.meshapi.ai/v1/chat/completions \  -H "Authorization: Bearer $MESH_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "auto",    "messages": [{"role": "user", "content": "Hello!"}]  }'

4. Move your config out of YAML

You don't keep a litellm_config.yaml anymore. Provider keys, model allowlists, rate limits, per-team budgets, RBAC | all of it lives in the Mesh dashboard. Edit in the browser, takes effect immediately, no deploy.

5. Decommission the proxy

Once requests are flowing through Mesh, point all remaining clients at the new base URL, watch the dashboard fill up, and shut the LiteLLM container down. Reclaim the box.

Streaming, tool calls, and multimodal

Same shape as LiteLLM. Streaming works with stream=True. Tool / function calling works on every model that supports it on the underlying provider. Vision inputs pass through unchanged.

stream = client.chat.completions.create(    model="auto",    messages=[{"role": "user", "content": "Write a short poem."}],    stream=True,)for chunk in stream:    if chunk.choices[0].delta.content:        print(chunk.choices[0].delta.content, end="")

What changes after migration

• No servers to babysit. We handle scaling, retries, key rotation, and provider failover.
• 40% lower blended cost. Negotiated rates plus smart routing to the cheapest model that meets the bar.
• Per-request visibility. Token counts, cost, latency, model used, error code | all real-time, all per-key.
• Per-team budgets and SSO. RBAC, audit log, per-key spend limits, ZDR | without writing any of it yourself.
• New models in 48 hours. No redeploy, no config push.

Common questions from LiteLLM teams

"Can I keep self-hosting?" Yes | if that's the reason you're on LiteLLM, stay on LiteLLM. Mesh is a managed product. If you want the managed version with negotiated rates and smart routing, that's us.

"Will my existing model strings work?" Yes. gpt-4.1, claude-opus-4-7, gemini-2.5-pro, deepseek-v3, qwen-3-max | all map directly. Use the same strings.

"What about my callbacks and custom logging?" Mesh emits webhook events for every request and exposes a usage API. Most teams turn off their custom logging within a week.

"Do you support Bedrock / Vertex / Azure OpenAI?" Yes, including provider-prefixed routing if you want to pin to a specific deployment.