LiteLLM and Docker let you build a self-hosted AI model gateway that routes application requests to multiple large language models via a single OpenAI-compatible API. Instead of connecting each application directly to OpenAI, Anthropic, Azure OpenAI, or another provider, you can send requests to a LiteLLM proxy and manage model routing from one configuration file.<\/p>
The main steps are:<\/p>
Deploying LiteLLM with Docker creates a self-hosted AI model gateway that receives OpenAI-compatible API requests and routes them to the models defined in your configuration file. LiteLLM’s Docker setup runs the proxy on port 4000, loads model routing rules from a YAML file, and uses environment variables to keep provider API keys outside the container image. LiteLLM’s official Docker quick start also supports a Docker Compose setup with Postgres for proxy deployments that need database-backed features.<\/p>
First, create a project directory for the LiteLLM deployment:<\/p>
mkdir litellm-gateway\ncd litellm-gateway<\/pre>Then, create a configuration file named litellm_config.yaml:<\/p>
nano litellm_config.yaml<\/pre>Add the following configuration:<\/p>
model_list:\nmodel_name: gpt-4o-gateway\nlitellm_params:\nmodel: openai\/gpt-4o\napi_key: os.environ\/OPENAI_API_KEY\ngeneral_settings:\nmaster_key: os.environ\/LITELLM_MASTER_KEY<\/pre>This file tells LiteLLM which model names your applications can call and where LiteLLM should route those requests. The model_name value is the user-facing name passed by the client application, while litellm_params.model is the backend model string LiteLLM uses to route the request to the correct provider.<\/p>
In this example, an application sends requests to gpt-4o-gateway. LiteLLM receives that request and forwards it to openai\/gpt-4o.<\/p>
2. Add your environment variables<\/h3>
Create an .env file to store the API keys used by the Docker container:<\/p>
nano .env<\/pre>Add your provider key and LiteLLM master key:<\/p>
OPENAI_API_KEY=your-openai-api-key\nLITELLM_MASTER_KEY=sk-your-secure-master-key<\/pre>Replace both placeholder values with your actual keys. The master key protects the LiteLLM proxy, while the provider API key lets LiteLLM send requests to the configured model provider.<\/p>
Keeping keys in environment variables is safer than writing them directly into the YAML file. LiteLLM’s deployment documentation also recommends using environment variables or secrets for API keys instead of hardcoding them into the config file.<\/p>
3. Create a Docker Compose file<\/h3>
Create a docker-compose.yml file:<\/p>
nano docker-compose.yml<\/pre>Add the following configuration:<\/p>
services:\nlitellm:\nimage: docker.litellm.ai\/berriai\/litellm:main-latest\ncontainer_name: litellm-gateway\nenv_file:\n- .env\nvolumes:\n- .\/litellm_config.yaml:\/app\/config.yaml\nports:\n- \"4000:4000\"\ncommand: [\"--config\", \"\/app\/config.yaml\", \"--port\", \"4000\"]\nrestart: unless-stopped<\/pre>This Compose file starts the LiteLLM container, loads your environment variables, mounts the local configuration file into the container, and exposes the proxy on port 4000.<\/p>
For production deployments, pin the image to a specific version instead of using main-latest. Version pinning makes rollbacks easier and prevents unexpected behavior after image updates. LiteLLM’s deployment docs also recommend using versioned images or SHA digests for predictable environments.<\/p>
4. Start the LiteLLM container<\/h3>
Run the container with Docker Compose:<\/p>
docker compose up -d<\/pre>Check that the container is running:<\/p>
docker ps<\/pre>Then, view the LiteLLM logs:<\/p>
docker logs litellm-gateway<\/pre>The proxy is ready when LiteLLM starts listening on port 4000. LiteLLM’s proxy commonly runs on http:\/\/0.0.0.0:4000, which exposes the gateway inside the container and maps it to the host through Docker.<\/p>
5. Test the LiteLLM gateway<\/h3>
Send a test request to the \/chat\/completions endpoint:<\/p>
curl http:\/\/localhost:4000\/chat\/completions\n-H \"Content-Type: application\/json\"\n-H \"Authorization: Bearer sk-your-secure-master-key\"\n-d '{\n\"model\": \"gpt-4o-gateway\",\n\"messages\": [\n{\n\"role\": \"user\",\n\"content\": \"Explain what an LLM gateway does in one sentence.\"\n}\n]\n}'<\/pre>The model value must match the model_name in your litellm_config.yaml file. In this setup, the application calls gpt-4o-gateway, and LiteLLM routes the request to openai\/gpt-4o.<\/p>
A successful response confirms three things:<\/p>
A model gateway becomes more useful when it routes requests to more than one backend model. To add another model, update litellm_config.yaml:<\/p>
model_list:\nmodel_name: gpt-4o-gateway\nlitellm_params:\nmodel: openai\/gpt-4o\napi_key: os.environ\/OPENAI_API_KEY\nmodel_name: gpt-4o-mini-gateway\nlitellm_params:\nmodel: openai\/gpt-4o-mini\napi_key: os.environ\/OPENAI_API_KEY\ngeneral_settings:\nmaster_key: os.environ\/LITELLM_MASTER_KEY<\/pre>Restart the container so LiteLLM loads the updated configuration:<\/p>
docker compose restart<\/pre>Now your application can call either route:<\/p>
curl http:\/\/localhost:4000\/chat\/completions\n-H \"Content-Type: application\/json\"\n-H \"Authorization: Bearer sk-your-secure-master-key\"\n-d '{\n\"model\": \"gpt-4o-mini-gateway\",\n\"messages\": [\n{\n\"role\": \"user\",\n\"content\": \"Write a short product description for an AI gateway.\"\n}\n]\n}'<\/pre>This pattern lets you keep one API endpoint while routing requests to different models based on cost, performance, or application needs. LiteLLM’s gateway is designed to provide a unified OpenAI-compatible interface for 100+ LLMs, including OpenAI, Anthropic, Vertex AI, Bedrock, and others.<\/p>
7. Stop or update the deployment<\/h3>
To stop LiteLLM, run:<\/p>
docker compose down<\/pre>To pull a newer image and restart the gateway, run:<\/p>
docker compose pull\ndocker compose up -d<\/pre>Use this update flow for testing environments. For production environments, update to a specific LiteLLM image version, test the gateway, and then restart the container with the pinned version.<\/p>
How to deploy LiteLLM faster with Hostinger’s Docker template<\/h2>
Hostinger’s Docker template<\/a> lets you deploy LiteLLM without manually installing Docker, pulling the LiteLLM image, or writing the initial container setup yourself. The template launches LiteLLM on a VPS<\/a>, so you can focus on adding your provider API keys, configuring model routes, and connecting your application to the LiteLLM gateway.<\/p>
To deploy LiteLLM with Hostinger’s Docker template:<\/p>
\n
- Go to Hostinger’s LiteLLM Docker template<\/a> page.<\/li>\n\n\n\n
- Choose a VPS plan for the deployment.<\/li>\n\n\n\n
- Click Deploy<\/strong>.<\/li>\n\n\n\n
- Complete the VPS checkout or log in to your Hostinger account.<\/li>\n\n\n\n
- Wait for Hostinger to provision the VPS and install the LiteLLM template.<\/li>\n\n\n\n
- Open your VPS management panel and check the LiteLLM service details.<\/li>\n\n\n\n
- Add your LLM provider API keys and model routing configuration.<\/li>\n\n\n\n
- Test the LiteLLM proxy endpoint from your application or terminal.<\/li>\n<\/ol>
Hostinger’s LiteLLM template is useful when you want a faster deployment path than manually setting up Docker. The catalog page describes LiteLLM as an AI gateway for calling 100+ LLMs through the OpenAI format, while the Docker catalog is designed for one-click application deployment.<\/p>
After the template is deployed, LiteLLM continues to function as a self-hosted AI gateway. Your application sends requests to the LiteLLM proxy, which routes them to the provider and model defined in your configuration. For example, you can expose a single internal endpoint for your application, routing one model name to OpenAI and another to a lower-cost model for simpler tasks.<\/p>
The Hostinger template is the better option if you want to skip the initial server setup and start from a preconfigured VPS environment. The manual Docker method is better if you need full control over every Compose setting, container image version, network rule, and dependency from the beginning.<\/p>
How does LiteLLM route requests between models?<\/h2>
LiteLLM routes requests by matching the model value in an API request with a model_name entry in your LiteLLM configuration file. After LiteLLM finds the matching route, it forwards the request to the backend provider and model defined under litellm_params.<\/p>
For example, this configuration creates two model routes:<\/p>
model_list:\nmodel_name: gpt-4o-gateway\nlitellm_params:\nmodel: openai\/gpt-4o\napi_key: os.environ\/OPENAI_API_KEY\nmodel_name: gpt-4o-mini-gateway\nlitellm_params:\nmodel: openai\/gpt-4o-mini\napi_key: os.environ\/OPENAI_API_KEY<\/pre>With this setup, an application does not call openai\/gpt-4o directly. Instead, it sends a request to the LiteLLM proxy and uses the gateway model name:<\/p>
{\n\"model\": \"gpt-4o-gateway\",\n\"messages\": [\n{\n\"role\": \"user\",\n\"content\": \"Summarize this support ticket.\"\n}\n]\n}<\/pre>LiteLLM receives the request, matches it against the first route in model_list, and forwards it to openai\/gpt-4o. If the application sends gpt-4o-mini-gateway, LiteLLM uses the second route and forwards the request to openai\/gpt-4o-mini.<\/p>
This routing layer lets you change backend models without changing the application’s request format. For example, you can keep the same application endpoint and update the LiteLLM configuration when you want to switch from one provider to another.<\/p>
LiteLLM routing is useful for three common gateway patterns:<\/p>
\n
- Route by task type.<\/strong> Send complex reasoning, coding, or analysis tasks to a stronger model, and send simple summarization or rewriting tasks to a smaller model.<\/li>\n\n\n\n
- Route by provider.<\/strong> Keep one OpenAI-compatible API format while forwarding requests to different providers, such as OpenAI, Anthropic, Azure OpenAI, Vertex AI, or AWS Bedrock.<\/li>\n\n\n\n
- Route by cost or availability.<\/strong> Use different routes for expensive production tasks, cheaper background jobs, or backup models when a provider is unavailable.<\/li>\n<\/ol>
For example, a team could use gpt-4o-gateway for customer-facing chatbot responses and gpt-4o-mini-gateway for internal text classification. Both requests go through the same LiteLLM proxy, but each model name points to a different backend route.<\/p>
For larger deployments, LiteLLM can also support fallback and load balancing patterns. A fallback route sends traffic to another model if the primary model fails, while load balancing distributes requests across multiple deployments of the same model. These options help make an AI gateway more reliable when several applications depend on the same LLM infrastructure.<\/p>
In practice, LiteLLM model routing provides teams with a single control point for LLM traffic. Instead of updating every application when a model, provider, or API key changes, you update the LiteLLM configuration and keep the application connected to the same gateway endpoint.<\/p>
How to secure and manage your LiteLLM gateway<\/h2>
A LiteLLM gateway should be protected at both the application and server levels. At the application level, use a strong LiteLLM master key, store provider API keys in environment variables, and create separate virtual keys for each application, user, or team. This setup helps prevent direct exposure of your OpenAI, Anthropic, Azure, or other provider credentials.<\/p>
At the server level, secure the VPS that runs the LiteLLM Docker container. Start by changing the default SSH port, disabling direct root login, using SSH keys instead of passwords, and enabling a firewall to allow only required ports. For a LiteLLM deployment, this usually means allowing SSH, HTTP\/HTTPS if you use a reverse proxy, and the LiteLLM proxy port only when it must be reached directly.<\/p>
You should also monitor login attempts, keep server packages up to date, scan for malware, and regularly review user permissions. These practices reduce the risk of brute-force attacks, broken authentication, exposed credentials, and unauthorized server access. Hostinger’s VPS security guide explains these steps in more detail, including SSH hardening, firewall setup, Fail2Ban, malware scanning, backups, and server monitoring. For the full checklist, read Hostinger’s guide on <\/a>VPS security<\/a>.<\/p>
For production use, avoid exposing the LiteLLM proxy directly to the public internet without access control. Put the gateway behind a reverse proxy, use HTTPS, restrict inbound traffic where possible, and issue separate LiteLLM keys for each application. This makes it easier to rotate credentials, track usage, and limit the impact if one key is compromised.<\/p>
When should you scale LiteLLM beyond a single Docker container?<\/h2>
A single LiteLLM Docker container is enough for testing, small internal tools, and early production deployments with predictable traffic. This setup works well when one application uses the gateway, the request volume is low to moderate, and manual container restarts do not create a major availability issue.<\/p>
You should scale LiteLLM beyond a single container once the gateway becomes part of your core AI infrastructure. Common signs include higher request volume, multiple applications sharing the same proxy, frequent rate-limit errors, strict uptime requirements, or the need to track usage across users and teams.<\/p>
For larger deployments, LiteLLM can run with additional infrastructure such as a database, Redis, and multiple proxy instances. A database helps store proxy-related data, such as keys, teams, budgets, and usage records. Redis helps support load balancing and shared rate-limiting across multiple LiteLLM containers. Multiple LiteLLM instances improve availability because traffic can continue flowing if one container fails.<\/p>
A scaled LiteLLM setup usually follows this pattern:<\/p>
\n
- Run several LiteLLM proxy containers.<\/li>\n\n\n\n
- Put a load balancer or reverse proxy in front of them.<\/li>\n\n\n\n
- Use a shared database for persistent gateway data.<\/li>\n\n\n\n
- Use Redis for shared request tracking and load-balancing behavior.<\/li>\n\n\n\n
- Monitor latency, error rates, model usage, and spend.<\/li>\n<\/ol>
Do not add this complexity before you need it. Start with a single Docker container, confirm that routing, authentication, and provider keys work correctly, then scale when traffic or reliability requirements warrant it.<\/p>
Once LiteLLM handles production traffic, treat it like any other critical VPS service. Keep the container image pinned, back up configuration files, rotate keys regularly, and monitor both the server and the gateway. This gives your AI model gateway a stable base for routing requests across providers while keeping future scaling predictable.<\/p>\n","protected":false},"excerpt":{"rendered":"
LiteLLM and Docker let you build a self-hosted AI model gateway that routes application requests to multiple large language models via a single OpenAI-compatible API. Instead of connecting each application directly to OpenAI, Anthropic, Azure OpenAI, or another provider, you can send requests to a LiteLLM proxy and manage model routing from one configuration file. […]<\/p>\n