Protect the client portal itself #

Before server creation, determine the management protection policy. Especially for team operations,
clarifying MFA and allowed IP handling first prevents later confusion about “who can access the management interface from where”.

Register SSH keys before creating servers #

Rather than inserting keys individually after AI-SRV and GPU worker creation, preparing keys beforehand in the client portal SSH key management
allows immediate management access after template deployment.

• Organize team operational public keys
• Don’t mix shared keys and individual keys in emergency situations
• Don’t post key fingerprints in published articles

Create AI-SRV on Bestnet Cloud from template #

Use Ubuntu 24 series template for AI-SRV, hosting both GPUStack Server and NFS.
Since this node also serves as model storage, design with sufficient storage area separate from the OS disk.

• Role is GPUStack Server + NFS
• Connect to private network
• Consider public side only if UI exposure is necessary
• Allocate more capacity for model storage on AI-SRV than on workers

Create workers on GPU-VPS in necessary quantities #

Since workers are for inference, select GPU-VPS plans based on GPU type, VRAM, and future model size.
On the other hand, since the model itself is kept on the NFS side, you can easily reduce worker storage to the minimum needed for OS and execution.

• Standardize on Ubuntu 24 series
• Create with GPU driver / CUDA available
• Connect to private network
• Consider private-only configuration if no public IP is needed

Place all nodes on 10G private network #

NFS and GPUStack internal communication flow over the 10G private link connecting Bestnet Cloud and GPU-VPS.
This allows model retrieval and worker join to remain on the private side.

• Accommodate AI-SRV and all workers in the same private segment
• Assign fixed IPs to stabilize subsequent NFS export and systemd settings
• If workers need outbound communication, choose NAT router method or temporary public egress

If you want to push workers to private-only, using Bestnet Cloud’s virtual router / NAT pattern for OS updates and external fetching only is easier to manage.

Determine initial firewall rules #

A policy of “allow only minimum necessary inbound to AI-SRV, and give workers almost no public inbound” is sufficient.
In the portal firewall screen, first check existing rules, then add only necessary ports.

With NFS v4 assumed, it’s easy to consolidate to 2049/tcp, though some environments may use rpcbind. Always limit public scope to private subnet.

Take initial snapshot #

After OS initial creation, key injection, private network connection, and firewall setup are complete,
take snapshots of AI-SRV and GPU workers to make it easier to restore if middleware installation fails later.

Install GPUStack Server #

In the initial build example, we deploy GPUStack Server on AI-SRV to enable login to the management UI.
After noting the initial admin password, move it to a safe vault and do not post in the article.

curl -sfL https://get.gpustack.ai | sh -s -
cat /var/lib/gpustack/initial_admin_password

# Web UI (example)
# http://10.10.0.10/

Create NFS shared directory #

Place the actual model files in /srv/gpustack_models.
Export this to the private subnet only, and also bind mount it as /models on AI-SRV itself.
This makes the path visible to the server and the path visible to workers consistent.

sudo mkdir -p /srv/gpustack_models

echo "/srv/gpustack_models 10.10.0.0/24(rw,sync,no_subtree_check,no_root_squash)" |   sudo tee -a /etc/exports

sudo exportfs -ra

# Server itself also bind mount
sudo mkdir -p /models
sudo mount --bind /srv/gpustack_models /models
echo "/srv/gpustack_models /models none bind 0 0" | sudo tee -a /etc/fstab

Issue worker join token #

Create a worker join token from the GPUStack GUI. In public articles, express as <WORKER_JOIN_TOKEN>
and do not post actual values. Also, limiting GUI login sources to the private network side is more secure.

First mount /models via NFS #

Each worker mounts the shared area exported by AI-SRV at the same path /models.
This makes the cache path for all workers consistent, preventing duplicate model storage.

sudo mkdir -p /models
echo "10.10.0.10:/srv/gpustack_models /models nfs defaults 0 0" | sudo tee -a /etc/fstab
sudo mount -a

After mounting, verify with mount | grep /models that the intended shared location is visible.

Install GPUStack Worker #

Use the token issued on the AI-SRV side to have each GPU-VPS join the cluster as a worker.

curl -sfL https://get.gpustack.ai | sh -s -   --server-url http://10.10.0.10   --token <WORKER_JOIN_TOKEN>

Fix --cache-dir /models in worker service #

In shared cache configuration, this specification is most important. Add
--cache-dir /models to the GPUStack Worker systemd definition and also fix the data-dir.

[Service]
ExecStart=/root/.local/bin/gpustack start   --server-url http://10.10.0.10   --token <WORKER_JOIN_TOKEN>   --cache-dir /models   --data-dir /var/lib/gpustack-data

sudo systemctl daemon-reload
sudo systemctl enable --now gpustack

# If "Using cache dir: /models" appears, OK
journalctl -u gpustack -f

Verify workers are Ready in GPUStack GUI #

In the GUI Workers screen, verify all GPU-VPS nodes are Ready.
If they remain Not Ready, troubleshoot in order: GPU driver / CUDA, NFS mount, worker logs.