> For the complete documentation index, see [llms.txt](https://docs.stg.ai.neevcloud.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stg.ai.neevcloud.com/gpu-ai-service/deploying-gpu.md).

# Deploying a GPU

NeevCloud GPU AI Service provides GPU Pods for AI/ML workloads, including model training, fine-tuning, inference, and image generation.

This document describes GPU Pod capabilities, available configuration options, pricing plans, networking, SSH access, Network Storage, and resource management.

## Prerequisites

* A NeevCloud account.
* Sufficient account credits for the selected configuration.
* An SSH key if SSH Terminal Access is required.

## GPU AI Service Overview

GPU AI Service enables users to select GPU resources and configure a software environment for their workloads. Deployment options include GPU selection, templates, pricing plans, networking, SSH access, and Network Storage.

### GPU Selection

GPU configurations can be evaluated based on workload requirements, resource availability, and budget.

Available filters include:

* **Region:** Deployment region.
* **VRAM:** GPU video memory capacity.
* **Brand:** GPU manufacturer.
* **Price:** Available configurations within the required budget.

Each GPU card displays details such as GPU model, VRAM, CPU, RAM, hourly price, and availability.

GPU models, prices, and available quantities depend on current availability.

## GPU Pod Configuration

### Pod Name

A deployment name is generated automatically. Users can retain the default name or edit it to identify their GPU Pod more easily.

Descriptive names are useful when managing multiple workloads.

### Templates

Templates define the software environment available in a GPU Pod. Users can browse templates, search by name, filter by category, and review template descriptions before selection.

Available templates shown in the console include:

* **PyTorch Ubuntu:** For deep learning workloads.
* **TensorFlow CUDA:** For TensorFlow workloads.
* **vLLM Inference:** For high-throughput LLM inference.
* **Triton Inference Server:** For model serving.
* **ComfyUI:** For image and video generation workflows.
* **H2O LLM Studio:** For LLM fine-tuning and training visualization.

The available templates may change. Review the current template details before deployment.

#### Common Template Scenarios

**Fine-tuning a Hugging Face model**

* **Recommended environment:** A PyTorch template with Transformers and PEFT.
* **Purpose:** PyTorch provides the deep learning framework, Transformers supports Hugging Face models, and PEFT provides parameter-efficient fine-tuning methods.

**Training a computer vision model**

* **Recommended environment:** A TensorFlow template with OpenCV.
* **Purpose:** TensorFlow supports model training, while OpenCV provides image-processing capabilities.

**Generating images with Stable Diffusion**

* **Recommended environment:** ComfyUI or another suitable Stable Diffusion template.
* **Purpose:** ComfyUI provides a visual workflow interface for building and running image-generation pipelines.

**LLM inference serving**

* **Recommended environment:** vLLM Inference.
* **Purpose:** vLLM is designed for high-throughput LLM inference and provides an API-based model-serving environment.

These are workload-based recommendations. Exact template names, included packages, and versions may differ. Verify the current template details in the console.

### Pricing Plans

GPU Pods support pricing options based on usage requirements.

* **On-Demand:** Pay for usage at the displayed hourly rate.
* **Savings Plan:** Select an available commitment duration to receive a discounted hourly rate.

The console may offer one-month, three-month, six-month, and one-year Savings Plans.

Review the hourly rate, commitment cost, and displayed savings before selecting a plan.

### Networking

Networking controls allow users to expose TCP and UDP ports for applications running on a GPU Pod.

#### TCP Ports

The following ports are displayed by default:

* **Port 22 (TCP):** Required for SSH access.
* **Port 8888 (HTTP):** Template port.

Additional TCP ports can be configured by entering a port number, selecting TCP, and clicking **Add TCP Port**.

Up to **3 custom TCP ports** can be exposed in addition to the selected template ports.

#### UDP Ports

UDP configuration supports low-latency traffic, including real-time voice applications using VoIP, SIP, or RTP.

Users can configure a single UDP port or a port range using the **Start port** and optional **End port** fields.

UDP configuration limits:

* **Allowed port range:** `1024–65535`
* **Maximum:** 5 port ranges.

Only the ports required by the application should be configured.

### SSH Terminal Access

SSH Terminal Access provides secure terminal connectivity to a GPU Pod.

Available SSH key options include:

* **Select SSH Key:** Select a saved key.
* **Paste SSH Key:** Paste a public SSH key.
* **Upload SSH Key:** Upload a public key file.
* **Generate SSH Key:** Generate a new SSH key.

Saved keys can be managed under **Account → SSH Keys**.

#### Finding an Existing SSH Public Key

On Linux or macOS:

```bash
cat ~/.ssh/id_rsa.pub
```

On Windows PowerShell:

```powershell
type $env:USERPROFILE\.ssh\id_rsa.pub
```

#### Generating an SSH Key

Users without an existing key can generate one using:

```bash
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
```

Replace `your_email@example.com` with your email address. Follow the prompts and press Enter to accept the default file location if appropriate.

The command creates two files:

* **Private key:** `~/.ssh/id_rsa`
* **Public key:** `~/.ssh/id_rsa.pub`

**Security note:** Keep the private key secure and never share it. Only the public key should be added to NeevCloud.

#### Adding an SSH Public Key

To configure SSH access, enable **SSH Terminal Access** in the GPU deployment configuration and use one of the available key options. New keys can also be saved under **Account → SSH Keys** for future use.

Ensure that the complete public key is provided without missing characters. The corresponding private key is required when connecting through SSH.

## Network Storage

Network Storage allows users to retain important data independently of the GPU Pod's lifecycle.

The available configuration fields include:

* **Name:** Name of the Network Storage volume.
* **Existing Network Storage:** Select an existing volume to reuse it.
* **Size (GB):** Required storage capacity within the displayed limits.
* **Mount Path:** Location where the storage is mounted inside the GPU Pod. The default shown in the console is `/data`.

Review the displayed storage allowance, billing information, and total storage cost before proceeding.

**Important:** Store datasets, model checkpoints, and other important files in the mounted Network Storage location. Files saved elsewhere on the GPU Pod may be lost when the Pod is deleted.

## Deployment and Access

### Deployment

Before deploying a GPU Pod, review the selected GPU configuration, template, pricing plan, networking, SSH settings, Network Storage, and Pricing Summary.

Select **Deploy GPU Pod** to start deployment. Follow the on-screen status and instructions to complete the process.

### Accessing a GPU Pod

After deployment, users can open the relevant GPU Pod view in the console and use the access options available for the selected configuration.

#### SSH Connection

When SSH Terminal Access is enabled and configured, users can connect using the provided connection details and corresponding private key.

SSH access supports:

* Running commands and scripts.
* Installing or configuring application-level dependencies.
* Managing files and processes.
* Running training and inference workloads.
* Monitoring execution and troubleshooting issues.

#### Web Interface

Templates that provide a web interface can be accessed through the corresponding console option, with networking configured as required.

Depending on the selected template, this may include tools such as Jupyter or ComfyUI. Availability depends on the selected environment.

#### GPU Availability Verification

When terminal access is available, GPU information and utilization can be checked using:

```bash
nvidia-smi
```

For a PyTorch environment, GPU availability can be verified using:

```python
import torch

print(torch.cuda.is_available())

if torch.cuda.is_available():
    print(torch.cuda.get_device_name(0))
```

These checks help verify that the environment can access the GPU.

## Managing GPU Pods

The GPU AI Service console provides options to review and manage deployed GPU Pods. Available actions depend on the current Pod status and console options.

### Resource Monitoring

Resource utilization helps users understand workload performance and identify bottlenecks.

Where available, monitor:

* GPU utilization.
* VRAM usage.
* CPU utilization.
* System memory usage.
* Network activity.
* Storage usage.

These metrics can help determine whether the selected configuration meets workload requirements.

### Stopping or Deleting a GPU Pod

Use the relevant management action in the console when a GPU Pod is no longer required.

Before deleting a Pod:

* Save important results and logs.
* Verify that required files are stored in Network Storage or another suitable location.
* Confirm that the Pod is no longer needed.

Deleting a Pod may remove data stored outside Network Storage. Review the console's confirmation message before proceeding.

Stopping and deleting are different actions. Check the applicable billing and resource-release behavior in the console before choosing an action.

## Automatic Environment Setup

When your GPU instance boots, NeevCloud automatically launches services based on your selected template. This automatic setup provides immediate benefits:

### Jupyter Notebook auto-launch

For templates including Jupyter, the notebook server starts automatically on boot. This means:

* No manual configuration needed
* Immediate access via the dashboard "Jupyter" button
* Pre-configured to work with the GPU
* Common libraries already imported in the environment

You can start working immediately:

```python
import torch
print(torch.cuda.is_available())  # Confirms GPU access
print(torch.cuda.get_device_name(0))  # Shows your GPU model
```

### ComfyUI auto-launch

For generative AI templates, ComfyUI starts automatically, providing:

* Visual workflow editor for diffusion models
* Pre-loaded model nodes and components
* Immediate image generation capability
* No command-line setup required

### Development environment ready

All templates include:

* GPU drivers loaded and initialized
* CUDA environment variables set correctly
* Framework-specific optimizations enabled
* Common directories created (`/workspace`, `/data`, `/models`)

## Deployment Recommendations

### Match Templates to Workloads

Choose a specialized template when it meets your requirements. A preconfigured environment with the required libraries can reduce manual setup.

### Verify CUDA Compatibility

Check application CUDA and framework requirements before deployment. Select a compatible template to reduce dependency and compatibility issues.

### Compare Pricing Plans

For extended GPU usage, compare available Savings Plans with On-Demand pricing. Consider the commitment duration, hourly rate, and total cost.

### Keep SSH Keys Organized

Use descriptive filenames for SSH keys associated with different projects.

For example:

```
~/.ssh/neevcloud_projectname_rsa
~/.ssh/neevcloud_production_rsa
```

Keep private keys secure and configure only the corresponding public keys in NeevCloud.

### Test Before Running Large Workloads

Run a small test workload before launching an expensive or long-running training job. Verify GPU availability, software dependencies, networking, and access to data.

## GPU Pod Management Best Practices

### Monitor Active GPU Pods

Review deployed GPU Pods regularly to identify unused resources, verify expected workloads, investigate performance issues, and identify unexpected costs.

### Use Meaningful Names

GPU Pod names are generated automatically, but users can use descriptive names where editing is supported.

Examples:

* **Project:** `gpt-finetuning-prod`
* **Purpose:** `training-run-experiment-5`
* **Team:** `research-team-nlp`

Meaningful names make it easier to identify workloads across multiple deployments.

### Manage Costs

For workloads with defined completion times:

* Monitor usage and estimated costs.
* Stop or delete resources when no longer required, as appropriate.
* Review the billing implications of each action.
* Confirm that important data has been saved before deleting a GPU Pod.

### Store Important Data in Network Storage

Use the configured Network Storage volume for datasets, model checkpoints, and other important files.

* Save important files in the configured mount path, such as `/data`.
* Verify that applications are writing to the correct directory.
* Keep additional copies of critical results in suitable external storage when required.
* Save logs and export results regularly.

Files saved outside Network Storage may be lost when the GPU Pod is deleted.

### Optimize GPU Selection

As workload requirements evolve:

* Measure actual resource utilization.
* Review GPU memory and compute requirements.
* Compare available GPU configurations.
* Choose a configuration that balances performance and cost.

Training and inference workloads can have different resource requirements. Select a configuration based on the actual workload rather than assuming that a larger GPU is always necessary.

### Manage Multiple GPU Pods

Organize GPU Pods according to their purpose.

**Organize by workload:**

* **Development:** Configurations suitable for coding and testing.
* **Training:** GPUs selected according to model size, memory requirements, and training duration.
* **Inference:** Configurations selected according to model requirements, expected traffic, and latency targets.

**Track costs:**

* Review usage and pricing for each workload.
* Organize GPU Pod names by project or purpose.
* Monitor costs across teams or projects where applicable.
* Consider available Savings Plans for workloads with predictable usage.

**Coordinate resource usage:**

* Plan long-running jobs to avoid unnecessary overlap.
* Share resources among team members when appropriate and supported.
* Consolidate compatible workloads where practical.
* Review active deployments regularly to avoid unnecessary resource consumption.