> For the complete documentation index, see [llms.txt](https://docs.stg.ai.neevcloud.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stg.ai.neevcloud.com/vm-service/vm-deployment.md).

# VM Deployment

This page covers the full technical detail of deploying a VM: all available instance configurations, storage sizing, startup scripts, and a complete step-by-step walkthrough. Use this page as your reference when selecting hardware and configuring your deployment.

## Step 1 - Select an Instance

This is the most important step. The instance you select determines your GPU/CPU hardware, available VRAM, RAM, and hourly cost. Choose based on your workload, see the tables below.

### CPU VM Configurations

CPU VMs provide virtual CPU cores and RAM with no GPU. They are the right choice for data preprocessing, ML pipeline orchestration, lightweight model serving, and any AI workflow that does not require CUDA.

| Size              | vCPUs   | RAM       | AI/ML Use Case                                                     |
| ----------------- | ------- | --------- | ------------------------------------------------------------------ |
| Small             | 2 vCPU  | 4 GB      | Smoke tests, CI runners, quick data checks                         |
| Medium            | 4 vCPU  | 8 GB      | Data preprocessing, model evaluation, pipeline APIs                |
| Large             | 8 vCPU  | 16 GB     | Airflow/Prefect orchestration, MLflow server, medium ETL           |
| XLarge            | 16 vCPU | 32 GB     | Large dataset preprocessing, distributed CPU inference, Spark jobs |
| 2XLarge / 4XLarge | 32 vCPU | 64–128 GB | Massively parallel CPU workloads, in-memory dataset processing     |

### GPU VM Configurations

GPU VMs pair CPU compute with dedicated NVIDIA GPU hardware. They are the right choice for model training, fine-tuning, LLM inference, diffusion models, and any AI workflow that requires CUDA acceleration.

| GPU Model                     | VRAM   | vCPU | RAM    | AI/ML Use Case                                                     |
| ----------------------------- | ------ | ---- | ------ | ------------------------------------------------------------------ |
| NVIDIA A30                    | 24 GB  | 24   | 192 GB | LLM inference up to 7B, QLoRA fine-tuning 7B–13B                   |
| NVIDIA GeForce RTX 5090       | 32 GB  | 16   | 96 GB  | LLM inference 7B–13B, QLoRA fine-tuning, Stable Diffusion / SDXL   |
| NVIDIA RTX A6000              | 48 GB  | 48   | 128 GB | LLM inference 7B–13B, full fine-tuning 7B, Stable Diffusion / SDXL |
| NVIDIA RTX PRO 6000 Blackwell | 96 GB  | 24   | 112 GB | QLoRA fine-tuning 30B–34B, large model inference                   |
| NVIDIA A100                   | 80 GB  | 12   | 125 GB | LLM inference 30B–70B, full fine-tuning 7B+                        |
| NVIDIA GeForce RTX 3090       | 24 GB  | 24   | 112 GB | LLM inference up to 7B, lightweight fine-tuning                    |
| NVIDIA H100                   | 80 GB  | 20   | 125 GB | LLM inference 30B–70B, large-scale pre-training                    |
| NVIDIA H200                   | 141 GB | 20   | 192 GB | Pre-training 70B+, multi-GPU large model workloads                 |
| Tesla T4                      | 16 GB  | 12   | 128 GB | LLM inference up to 7B, lightweight GPU workloads                  |

## Step 2 - Configure Deployment

After selecting your instance, configure the deployment environment. Fill in the following fields:

| Field             | Purpose                                                                                                                 | Example Value                    |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------- | -------------------------------- |
| VM Name           | A label to identify your instance in the dashboard.                                                                     | llama3-finetune-v1               |
| Instance Type     | Auto-filled from Step 1. Confirm it matches CPU or GPU.                                                                 | GPU VM                           |
| Compute Resources | Read-only summary of your Step 1 selection.                                                                             | RTX A6000 / 48 vCPU / 128 GB RAM |
| Default Storage   | Pre-attached 50 GB SSD root disk. Adjustable in Step 3.                                                                 | 50 GB SSD (increase in Step 3)   |
| Image             | Choose OS Image (fresh VM from a base OS) or Bootable Volume (reuse an existing saved disk). See Image Selection below. | Fine-Tuning template             |
| SSH Key           | Your public key for terminal access.                                                                                    | Paste \~/.ssh/id\_ed25519.pub    |

## Startup Scripts

A startup script runs automatically on the first boot of your VM, before you SSH in. Use it to automate your environment setup so you do not have to repeat manual steps every time you deploy a new instance.

**Example:** startup script for a QLoRA fine-tuning VM:

```bash
#!/bin/bash
set -e

# Install fine-tuning dependencies
pip install --upgrade transformers peft trl bitsandbytes accelerate datasets

# Clone your training repository
git clone https://github.com/your-org/finetune-repo.git /workspace/train

# Set Hugging Face cache location
echo 'export HF_HOME=/workspace/hf_cache' >> /root/.bashrc
echo 'export TRANSFORMERS_CACHE=/workspace/hf_cache' >> /root/.bashrc

# Download base model (set HF_TOKEN in environment or script)
huggingface-cli download meta-llama/Meta-Llama-3-8B \
  --local-dir /workspace/models/llama3-8b \
  --token ${HF_TOKEN}
```

## Step 3 - Configure Storage

Every VM starts with a 50 GB SSD root disk. For AI/ML workloads, you almost always need to increase this. Size your disk during Step 3, before deployment, you cannot resize the root disk after the VM is created without rebuilding it.

### Storage Sizing Guide

| What You Are Storing            | Typical Size | Notes                                  |
| ------------------------------- | ------------ | -------------------------------------- |
| OS + base ML framework          | \~20–30 GB   | Already on disk via AI template        |
| 7B model weights (FP16)         | \~14 GB      | e.g., LLaMA 3 8B, Mistral 7B           |
| 13B model weights (FP16)        | \~26 GB      | e.g., LLaMA 2 13B, Vicuna 13B          |
| 70B model weights (FP16)        | \~140 GB     | Requires large disk + high-VRAM GPU    |
| Training dataset (text, medium) | 1–50 GB      | Depends heavily on dataset size        |
| Fine-tuning checkpoints         | 5–30 GB each | Save only best N checkpoints           |
| Hugging Face model cache        | Varies       | Set HF\_HOME to control cache location |

> **Tip:** For most LLM fine-tuning jobs (7B–13B models), start with 200 GB. This covers the model, a medium dataset, several checkpoints, and the framework stack with headroom to spare.

## Post-Deployment: Connecting and Verifying

After deployment, your VM first shows Provisioning status while it starts up, then transitions to Running once ready. Once Running, the dashboard displays your Public IP and a ready-to-use SSH Command. Connect with SSH and run these verification commands:

```bash
# Connect to your VM
chmod 400 /path/to/key.pem
ssh -i /path/to/key.pem root@<your-public-ip>

# Check disk space
df -h

# Verify GPU is detected
nvidia-smi

# Verify PyTorch sees the GPU
python -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"

# Check CUDA version
nvcc --version
```

Keep long-running training jobs alive after you disconnect by using tmux:

```bash
# Start a tmux session before launching training
tmux new -s training

# Run your training script
python train.py --epochs 20 --output /workspace/checkpoints

# Detach from tmux (job keeps running): press Ctrl+B, then D

# Re-attach later:
tmux attach -t training
```

> **Warning:** Always run training jobs inside tmux or screen. If your SSH connection drops without tmux, the process is killed and your training run is lost.