warmctl
Checkpoint/restore for vLLM that packages a warmed-up engine (processes, GPU state with CUDA graphs, processed weights; 1–8 GPUs on one node) as an OCI artifact and restores it in seconds on hosts with the same GPUs and driver branch.
Checkpoint/restore for vLLM that snapshots a warmed engine and restores it in seconds on matching NVIDIA GPU hosts.
What it is
warmctl is an Apache-2.0 tool for vLLM that saves a fully warmed-up inference engine, including its processes, GPU state with CUDA graphs, and processed weights, as an OCI artifact. A new replica can restore that snapshot instead of loading weights, compiling, autotuning kernels and capturing CUDA graphs again. The project's own measurements compare a GLM-5.3-Flash cold start on 8×H100 at 650 seconds with a warmctl restore at 37 seconds. The snapshot works on 1 to 8 GPUs within one node, and the target host must match the GPUs and NVIDIA driver branch. The project is in beta and publishes releases as pre-releases.
Key features
- Restores a warmed vLLM engine, with its processes, GPU state and weights, on hosts with matching GPUs and driver branch
- Packages snapshots as OCI artifacts that can be pushed to any OCI registry or shared file system
- Checks the host before download, verifies SHA-256 for every file, and runs a canary prompt before serving
- Supports tensor and expert parallel setups on 1 to 8 GPUs within one node
- Includes a Kubernetes runtime image, KEDA autoscaler and KServe preset in deploy/kubernetes
Use cases
- Scaling replicas quickly so a new vLLM instance can start serving in seconds instead of minutes
- Scaling large models to zero and bringing them back without a long cold start
- Replacing spot GPU instances faster after an interruption
Who it's for
Teams running vLLM on NVIDIA GPU hosts or Kubernetes clusters who need fast replica startup and can accept beta software. It is less suited to multi-node engines or engines other than vLLM, which the README lists as not yet supported.
Getting started
The README gives Linux GPU host requirements (Docker with the NVIDIA runtime, root, driver 580+, and kernel.io_uring_disabled=2) and an install flow, but this summary does not reproduce its setup commands, so none are shown here. Start from the README's Requirements and Quickstart sections and the project's documentation links.
Alternatives
The provided directory lists Cog (Replicate) as a related entry. This is a directory association, not a claim about how the two compare.
Drafted with AI from the project's public documentation; verify against the source. Written 3 Nov 2026.