Identify legacy data dir and back up first #

We confirmed the legacy systemd definition to clarify which data dirs the Server and Worker use respectively.
Then we performed service stop → tar backup → permission adjustment in order.

sudo systemctl cat gpustack | sed -n '/ExecStart/p'
sudo systemctl stop gpustack
sudo systemctl disable gpustack

sudo tar -C / -cpf /root/backup/gpustack-server.tar var/lib/gpustack

Migrate management node first to v2.1.1 #

Since the management node was assumed to not use GPU, we omitted --runtime nvidia from Server startup.
If left attached, unknown or invalid runtime name: nvidia will fail startup
on servers where NVIDIA runtime is not registered in Docker.

sudo docker run -d --name gpustack \
  --restart=unless-stopped \
  --privileged \
  --network=host \
  --env GPUSTACK_DATA_MIGRATION=true \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /var/lib/gpustack:/var/lib/gpustack \
  --volume /models:/models \
  gpustack/gpustack:v2.1.1 \
  --disable-worker \
  --cache-dir /models

We explicitly specified --disable-worker because we wanted to cleanly separate roles
from the old embedded worker on the management node. The theme of this migration was “management node focuses on management”.

Gradually switch GPU workers to Docker-based worker #

Workers inherited the existing data dir while being reconnected as Docker-based workers.
To match shared cache paths, we bind /models:/models to the container as well,
and append --cache-dir /models to the startup arguments.

sudo docker run -d --name gpustack-worker \
  --restart=unless-stopped \
  --privileged \
  --network=host \
  --volume /var/lib/gpustack-data:/var/lib/gpustack \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /models:/models \
  --runtime nvidia \
  gpustack/gpustack:v2.1.1 \
  --server-url http://<SERVER_IP> \
  --token <JOIN_TOKEN> \
  --worker-name <WORKER_NAME> \
  --worker-ip <WORKER_IP> \
  --cache-dir /models

The key point is not treating the GUI “Add Worker” as simply a “registration task”.
In reality, the GUI is just an entry point to obtain tokens and execution commands;
Ready status comes only after executing the actual command on the worker node.

Verify NVIDIA Container Toolkit #

GPU workers require --runtime nvidia. If Docker doesn’t know the runtime,
NVIDIA Container Toolkit configuration may not be reflected. Follow the official procedure
to configure Docker runtime with nvidia-ctk and restart Docker.

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Clean up Workers list to “only nodes that actually perform inference” #

In the legacy environment, the management node remained as a worker, appearing as Not Ready in the list, which was a source of confusion.
Once the 2 GPU workers became Ready, we targeted stale workers from the management node for cleanup.
This makes the role intuitive even for operations teams.