{"id":148626,"date":"2026-09-07T05:01:47","date_gmt":"2026-09-07T05:01:47","guid":{"rendered":"\/ca\/tutorials\/deploy-llm-with-vllm"},"modified":"2026-09-07T05:01:47","modified_gmt":"2026-09-07T05:01:47","slug":"deploy-llm-with-vllm","status":"publish","type":"post","link":"\/ca\/tutorials\/deploy-llm-with-vllm\/","title":{"rendered":"How to deploy an LLM with vLLM on a GPU server"},"content":{"rendered":"

To deploy an LLM with vLLM, prepare your GPU server and install vLLM there, then load your preferred vLLM-compatible model and serve it through an OpenAI-compatible API<\/strong>.<\/p>

vLLM itself is a tool for running and serving LLMs on GPUs. It’s designed for efficient model serving, especially when several requests need to share the same GPU.<\/p>

Here’s how to deploy an LLM with vLLM on your GPU server:<\/p>

    \n
  1. Choose a GPU with enough VRAM for your model and expected workload.<\/li>\n\n\n\n
  2. Connect to the GPU server over SSH and verify the NVIDIA hardware.<\/li>\n\n\n\n
  3. Set up an isolated Python environment and install vLLM.<\/li>\n\n\n\n
  4. Load the model and start its OpenAI-compatible API.<\/li>\n\n\n\n
  5. Keep the vLLM server running with a systemd service.<\/li>\n\n\n\n
  6. Protect remote API access with a private port and SSH tunnel.<\/li>\n\n\n\n
  7. Send test requests and confirm the API key and network restrictions work.<\/li>\n\n\n\n
  8. Compare inference throughput with single and concurrent requests.<\/li>\n<\/ol>

    This tutorial uses Qwen2.5-Coder-7B-Instruct as the vLLM model and Hostinger GPU as the provider. The overall process applies to other vLLM-compatible models and GPU providers, with slight differences in GPU sizing, server setup, and control-panel menus.<\/p>

    <\/p>

    1. Choose a Hostinger GPU for the model<\/h2>

    Choose the L40S GPU with 48 GB of VRAM for the Qwen2.5-Coder-7B-Instruct model when you rent a GPU<\/a> from Hostinger.<\/p>

    Your GPU for vLLM needs enough VRAM for the model weights, which depend on the parameter count and precision, plus the KV cache, runtime overhead, context length, and concurrent requests.<\/p>

    Start with the model weights because they set the baseline VRAM requirement. Use this simple estimate for a model with BF16 weights, such as Qwen2.5-Coder-7B-Instruct<\/a>:<\/p>

    VRAM for model weights ≈ parameter count × 2 bytes<\/pre>
    The model has 7.61 billion parameters, so the calculation is:<\/p>
    7.61 billion × 2 bytes ≈ 15.2 GB<\/pre>
    Don’t treat 15.2 GB<\/strong> as the model’s total VRAM requirement, though.<\/p>
    You also need GPU memory for the CUDA runtime, activations that temporarily hold data while the model processes tokens, which stores previously processed token data for reuse during generation.<\/p>
    Hostinger offers the RTX 4090 with 24 GB of VRAM, starting at $0.38\/hour<\/strong>. That capacity exceeds the model’s estimated 15.2 GB weight footprint.<\/p>
    However, we recommend the L40S, starting at $0.92\/hour<\/strong>, because its 48 GB of VRAM provides substantially more room for the KV cache, runtime overhead, longer context windows, and concurrent requests.<\/p>
    Use this comparison table to choose the right Hostinger GPU<\/a> for your workload:<\/p>
    GPU<\/strong><\/td>VRAM<\/strong><\/td>When to choose it<\/strong><\/td><\/tr>
    RTX 4090<\/td>24 GB<\/td>Development, experimentation, and small- to medium-model inference<\/td><\/tr>
    L40S<\/strong><\/td>48 GB<\/strong><\/td>AI inference and generative AI workloads (recommended for this deployment<\/strong>)<\/td><\/tr>
    A100 80GB PCIe<\/td>80 GB<\/td>LLM inference with larger memory requirements, model training, and research workloads<\/td><\/tr>
    RTX PRO 6000 (Server)<\/td>96 GB<\/td>Large-model inference and fine-tuning that require more VRAM than the L40S provides<\/td><\/tr>
    B200<\/td>192 GB<\/td>Large-model fine-tuning, inference at scale, and other highly memory-intensive AI workloads<\/td><\/tr>
    B200 (Dedicated)<\/td>192 GB<\/td>The same workloads as the B200 when you specifically need dedicated, non-shared GPU resources<\/td><\/tr><\/tbody><\/table><\/figure>
    After choosing your GPU, set up your instance<\/a> in hPanel to choose your operating system, top up your account credits, and deploy it.<\/p>
    \"hPanel