vLLM vs TGI: TGI is in maintenance mode, so which should you deploy?

Quick verdict

For a new deployment, choose vLLM (or SGLang). TGI accepts only minor fixes now, its last release was v3.3.7 in December 2025, and its GitHub repository is archived, so it will not pick up new model architectures or performance work. If TGI already serves your traffic, it keeps working: there is no need to migrate in a hurry, but plan the move rather than building anything new on it.

Comparison matrix

PagedAttention: what vLLM does differently

The core vLLM architectural innovation is PagedAttention, a KV cache memory management system modeled on OS virtual memory paging. Traditional inference servers allocate contiguous blocks of GPU memory per request, which leads to fragmentation and limits the number of concurrent requests that can be batched. PagedAttention allows the KV cache to be stored in non-contiguous memory blocks and reallocated dynamically across requests.

In practice this matters most under two conditions: long-context requests where the KV cache per request is large, and high-concurrency workloads where many requests compete for the same GPU memory pool. Under heavy load, vLLM memory efficiency translates directly into higher sustainable throughput before CUDA out-of-memory errors occur. TGI later adopted paged attention kernels of its own, so the gap on this point narrowed; what separates the projects now is that only one of them is still moving.

TGI today: maintenance mode

Hugging Face's own notice in the TGI repository says the project is in maintenance mode: it accepts pull requests for minor bug fixes, documentation and lightweight maintenance only. The same notice explains why. TGI helped move optimized inference engines onto the shared transformers model architectures, that approach is now used by downstream engines, and Hugging Face contributes to and recommends vLLM and SGLang going forward, alongside local engines such as llama.cpp.

For you, that means TGI still runs the models it already supports, but new model families, new quantization formats and new hardware generations will arrive in vLLM and SGLang, not in TGI. If you use Hugging Face Inference Endpoints, check which engine your endpoint runs before assuming it is TGI.

Licensing: no longer a difference

Both projects are Apache 2.0. Older comparisons, including an earlier version of this page, describe TGI as restricted for commercial inference-as-a-service use. That was true of the Hugging Face Optimized Inference License TGI adopted in July 2023, but Hugging Face reverted TGI to Apache 2.0 with version 2.0.0 in April 2024. The restriction only applies if you still run a pinned 1.x release, which is a reason to upgrade on its own.

Choose vLLM when

• You are starting a new self-hosted deployment of an open model.
• You need support for model families released after TGI stopped taking feature work.
• Your workload involves long-context requests or very high concurrency where PagedAttention memory management delivers measurable throughput gains.
• You need FP8 quantization for NVIDIA Hopper GPUs (H100, H200) or advanced speculative decoding.
• You want an engine that will keep getting performance and security work.

Keep TGI for now when

• It already serves production traffic, the models you run are supported, and nothing is forcing a change.
• Your clients depend on TGI's own /generate API and you have not yet budgeted the client changes.
• You have tuned launcher settings and monitoring around TGI and want to migrate on your own schedule.

Moving an existing TGI deployment to vLLM

1. List every client and what it calls. Clients on the OpenAI-compatible chat completions route need little change; clients on TGI's /generate or /generate_stream endpoints need rewriting.
2. Map the launcher settings: TGI's --num-shard becomes vLLM's --tensor-parallel-size, --quantize becomes --quantization, and the TGI token limits (--max-input-tokens, --max-total-tokens) map onto vLLM's --max-model-len.
3. Confirm your exact model and quantization load in vLLM before anything else; a format TGI accepted is not guaranteed to be one vLLM accepts.
4. Run both side by side on a copy of real traffic and compare latency percentiles, throughput and error rates, not a synthetic benchmark.
5. Point your Prometheus scraping at vLLM's /metrics endpoint and rebuild dashboards; the metric names differ.
6. Shift traffic gradually and keep the TGI deployment available for rollback until the new one has run cleanly under peak load.

Check your hardware first

Before choosing an inference server architecture, confirm your GPU VRAM and system RAM can support your target model and quantization level. Throughput comparisons between engines are meaningless if the hardware cannot hold the model without swapping.

Limitations of this comparison

• vLLM releases frequently, so quantization support, multi-GPU strategies and latency characteristics change across versions.
• We have not benchmarked either server on our own hardware for this page; the comparison is based on the projects' documentation and status.
• Hardware generation matters significantly: H100 (FP8 native) vs A100 vs consumer RTX GPUs produce very different throughput curves on the same model and server.
• Project status can change. Check the TGI repository notice and the vLLM release history before deciding.

Sources