> For the complete documentation index, see [llms.txt](https://docs.stg.ai.neevcloud.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.stg.ai.neevcloud.com/gpu-ai-service/faq.md). # FAQ ## Understanding GPU AI Service
What is NeevCloud GPU AI Service? NeevCloud GPU AI Service provides you with ready-to-use GPU environments specifically designed for AI and machine learning workloads. The service runs GPUs inside containerized pods with pre-configured AI frameworks, libraries, and development tools. This means you get immediate access to production-ready environments without spending time on infrastructure setup.
What is the primary purpose of GPU AI Service? GPU AI Service is designed for running AI and machine learning workloads in pre-configured environments. The primary use cases include: * **Model training**: Train deep learning models with PyTorch, TensorFlow, or other frameworks * **Model fine-tuning**: Fine-tune pre-trained models like LLMs or vision models * **Inference and serving**: Deploy models for predictions and real-time inference * **AI experimentation**: Run experiments in Jupyter notebooks with GPU acceleration * **Generative AI workflows**: Create images, text, or other content using diffusion models or LLMs * **Data processing**: Perform GPU-accelerated data transformations and analytics The service is optimized for AI/ML workloads, not for general-purpose computing or hosting arbitrary applications.
*** ## GPU Provisioning and Limits
How many GPUs can I provision per deployment? You can provision a maximum of 8 GPUs per deployment. If you are looking for more than 8 GPUs, please refer our GPU Cluster Service. **Single GPU deployments**: Ideal for most training, fine-tuning, and inference workloads **Multi-GPU deployments (2-8 GPUs)**: Use for distributed training, large model training, or high-throughput inference serving
Can I provision more than 8 GPUs for a single deployment? No, the maximum limit is 8 GPUs per deployment for GPU AI Service. If your workload requires more than 8 GPUs, please refer our GPU Cluster Service.
What happens if I need to scale beyond 8 GPUs? For workloads requiring more than 8 GPUs, please refer our GPU Cluster Service.
*** ## Data Persistence and Storage
What data is retained when I stop or restart my instance? **Only data stored in the `/data` directory is retained**. This directory is backed by network storage that persists across instance restarts and stops. **Retained (stored in `/data`)**: * Datasets you upload to `/data` * Model checkpoints saved to `/data` * Training logs written to `/data` * Any files explicitly saved to `/data` **Not retained (stored elsewhere)**: * Files in `/workspace`, `/root`, `/tmp`, or other directories * Software packages installed with pip or apt-get * System configurations or environment modifications * Files in any directory except `/data`
Where should I store my datasets and model checkpoints? Always store important data in the `/data` directory. This is the only location backed by persistent network storage. **Best practice structure**: ``` /data/ ├── datasets/ # Your training datasets ├── models/ # Saved model checkpoints ├── outputs/ # Training outputs and results ├── logs/ # Training logs └── code/ # Your code (if not version controlled) ``` **Example in your training script**: ```python # Save checkpoints to /data for persistence checkpoint_path = "/data/models/my_model_checkpoint.pth" torch.save(model.state_dict(), checkpoint_path) # Load datasets from /data dataset_path = "/data/datasets/my_training_data/" dataset = load_dataset(dataset_path) ```
What happens to data outside `/data` when I restart my instance? All data stored outside the `/data` directory is lost when your instance is restarted or deleted. This includes: * Python packages installed with pip * System packages installed with apt-get * Files in your home directory (`/root`) * Files in `/workspace` or `/tmp` * Any configuration changes you made After a restart, your instance returns to the original template state, with only the data in `/data` preserved.
Can I use external storage like S3 or cloud storage? Yes, you can connect to external storage services from your instance. This is recommended for: * **Large datasets**: Store in S3, Google Cloud Storage, or Azure Blob Storage and download as needed * **Backup**: Regularly sync important data to external storage * **Collaboration**: Share data across multiple instances or team members * **Archival**: Move completed project data to cheaper long-term storage **Example using AWS S3**: ```bash # Install AWS CLI (remember: not retained after restart) pip install awscli # Download dataset from S3 to /data aws s3 sync s3://my-bucket/datasets/ /data/datasets/ # Upload results back to S3 aws s3 sync /data/models/ s3://my-bucket/trained-models/ ```
How much storage space do I have in `/data`? Storage capacity in `/data` depends on storage size you selected during deployment. First 100 GB is free for Network Storage. For more than 100 GB, you will be charged as per the pricing model. Check your instance details in the dashboard to see your allocated storage capacity.
*** ## Software Installation and Customization
Can I install additional software or Python packages? Yes, you can install additional software during your session, but **installations are not retained after restart or deletion**. When your instance restarts, all installed packages are lost, and you return to the original template state. **What you can install (temporarily)**: ```bash # Install Python packages pip install transformers accelerate # Install system packages apt-get update && apt-get install vim # Install development tools pip install pytest black flake8 ``` **What happens after restart**: All these installations are gone, and you need to reinstall them.
Why aren't my installed packages retained after restart? GPU AI Service uses containerized pods designed for AI workloads with pre-configured templates. This architecture provides: * **Consistency**: Every restart gives you a clean, known-good environment * **Security**: No accumulated cruft or security vulnerabilities * **Performance**: Faster provisioning without loading custom installations * **Reliability**: Reduces configuration drift and environmental issues If you frequently need custom packages, consider these alternatives: 1. **Use a different template**: Choose a template that already includes your needed packages 2. **Use BYO AI Template** (coming soon): Bring your own custom environment configuration 3. **Create installation scripts**: Write a script to quickly reinstall packages at startup 4. **Request new templates**: If you frequently need the same packages, request a new template
What if I need custom software every time I start my instance? Create an initialization script that installs required packages automatically when your instance starts: **Create `/data/setup.sh`**: ```bash #!/bin/bash # This script is stored in /data, so it persists echo "Installing custom packages..." pip install transformers accelerate bitsandbytes pip install wandb tensorboard echo "Setting up environment..." export WANDB_API_KEY="your-key-here" echo "Setup complete!" ``` **Run on each startup**: ```bash # After SSH into your instance bash /data/setup.sh ``` Since this script is in `/data`, it persists across restarts. However, you still need to run it manually after each restart.
Should I use the plain Ubuntu AI template? **Generally, no**. The plain Ubuntu AI template requires significant manual setup and none of your installations persist after restart. This creates extra work every time you start your instance. **Instead, you should**: 1. **Choose a specialized template**: Use templates like PyTorch, TensorFlow, or Stable Diffusion that match your workload 2. **Wait for BYO AI Template**: If no template fits your needs, the upcoming "Bring Your Own AI Template" feature will let you define custom environments 3. **Request a new template**: Contact support to request a template for your specific use case **Only use plain Ubuntu AI if**: * You're doing a one-time experiment * You're testing something not related to standard AI frameworks * You explicitly need a minimal environment For repeated work, specialized templates save significant time and effort.
*** ## Container and Virtualization Support
Can I run Docker or containers inside my GPU AI Service? **No, running nested containers is not supported**. GPU AI Service already runs your workload inside a containerized pod. Running Docker or other container runtimes inside your instance creates nested containerization, which is not allowed. **Why nested containers aren't supported**: * **Architecture limitation**: Your instance is already a container, nesting isn't supported * **Performance degradation**: Nested containers add overhead and reduce GPU performance * **Security concerns**: Nested containers create additional security attack surfaces * **Resource conflicts**: Container runtimes conflict with the pod's resource management
What if I need to use Docker for my workflow? If your workflow requires Docker or containerization, GPU AI Service may not be the right fit. Consider these alternatives: **For AI/ML workloads without Docker**: * Use the pre-configured templates that match your framework * Adapt your workflow to run directly in the instance without containers * Use virtual environments (venv, conda) instead of containers for isolation **For workflows requiring containerization**: * Use NeevCloud VM Service designed for container workloads * Use NeevCloud Kubernetes Service (NKS) for container orchestration * Contact support to discuss your requirements
I installed Docker but it's not working. Why? Docker and similar container runtimes are not supported in GPU AI Service instances. Even if you install Docker, it won't function properly because: * The instance lacks necessary kernel capabilities for running containers * GPU access doesn't work correctly in nested containers * The instance architecture prevents container runtime operations **Solution**: Remove Docker and adapt your workflow to run directly in the instance, or consider alternative NeevCloud services (VM, NKS) for containerized workloads.
Can I use Kubernetes or container orchestration tools? No, Kubernetes, Docker Swarm, and other container orchestration tools are not supported inside GPU AI Service. Your instance is a single containerized pod managed by NeevCloud's infrastructure. If you need container orchestration, explore NeevCloud's other services (VM, NKS) designed for those use cases.
I installed software with apt-get but systemctl commands don't work. Why? GPU AI Service instances run inside containers without systemd support. This means `systemctl` commands will fail even after successfully installing software packages. **What doesn't work**: ```bash # These commands will fail sudo systemctl start nginx sudo systemctl enable postgresql sudo systemctl status redis-server systemctl restart docker ``` **Why this happens**: * Containers don't include the systemd init system * Service management commands like `systemctl`, `service`, and `systemd` are not available * The containerized environment uses a different process management approach
*** ## Instance Lifecycle and Restarts
When does my instance restart? Your instance may restart in several scenarios: * If the main container process exits or crashes * During infrastructure maintenance or updates * If the instance becomes unresponsive or unhealthy **Important**: After any restart, you lose all data outside `/data` and all software installations.
What happens when my instance restarts? When your instance restarts, it returns to a clean state based on your selected template: **What's retained**: * Data in `/data` directory * Your SSH keys (you can still connect) * Instance configuration (GPU type, template, etc.) **What's lost**: * All installed Python packages * All installed system packages * Running processes and training jobs * Files outside `/data` * Environment variables you set * System configuration changes
How can I prevent unexpected data loss during restarts? Follow these best practices: 1. **Save everything to `/data`**: Always store important files in `/data` 2. **Use tmux or screen**: Keep long-running processes in persistent sessions 3. **Implement checkpointing**: Regularly save model checkpoints to `/data` 4. **Monitor your instance**: Check dashboard alerts for planned maintenance 5. **Use external backup**: Regularly sync critical data to S3 or cloud storage **Example checkpointing in training**: ```python import os # Save checkpoint every N epochs to /data checkpoint_dir = "/data/models/checkpoints/" os.makedirs(checkpoint_dir, exist_ok=True) for epoch in range(num_epochs): train_one_epoch() # Save checkpoint to persistent storage if epoch % 5 == 0: checkpoint_path = f"{checkpoint_dir}/model_epoch_{epoch}.pth" torch.save({ 'epoch': epoch, 'model_state_dict': model.state_dict(), 'optimizer_state_dict': optimizer.state_dict(), 'loss': loss, }, checkpoint_path) print(f"Checkpoint saved to {checkpoint_path}") ```
Can I prevent my instance from restarting? You cannot prevent all restarts, especially those triggered by infrastructure maintenance. However, you can: * **Monitor health**: Keep your workloads stable to prevent crash-triggered restarts * **Plan for restarts**: Design your workflows to be restart-tolerant * **Use persistent storage**: Always save work to `/data`
*** ## Use Cases and Best Practices
What are the ideal use cases for GPU AI Service? GPU AI Service is perfect for: **Model Training**: * Training neural networks from scratch * Transfer learning with pre-trained models * Hyperparameter tuning and experimentation * Training computer vision, NLP, or multimodal models **Model Fine-Tuning**: * Fine-tuning large language models (LLMs) * Adapting pre-trained models to specific domains * Parameter-efficient fine-tuning (LoRA, QLoRA, PEFT) * Few-shot learning with custom datasets **Inference and Deployment**: * Running inference on trained models * Batch prediction on large datasets * Real-time inference serving with APIs * Model evaluation and benchmarking **Research and Experimentation**: * Prototyping new architectures * Reproducing research papers * Running experiments in Jupyter notebooks * Exploring new AI techniques **Generative AI**: * Image generation with Stable Diffusion * Text generation with LLMs * Audio and video generation * Multi-modal content creation
What are GPU AI Service instances NOT suitable for? GPU AI Service is not designed for: **General-purpose computing**: * Web servers or application hosting * Database servers * General file storage * Non-AI computational workloads **Container orchestration**: * Running Docker or Kubernetes * Container-based microservices * Nested containerization * Service mesh deployments **Long-term persistent environments**: * Development environments that need custom installations retained * Environments requiring specific system configurations preserved * Workflows dependent on persistent file systems outside `/data` **Traditional software development**: * Compiling and building large software projects (unless AI-related) * Running CI/CD pipelines * Software testing frameworks (unless for AI models)
How should I structure my workflow to work best with GPU AI Service? Follow this recommended workflow: **1. Preparation (before deployment)**: * Choose the appropriate template for your framework * Prepare your datasets and upload to external storage * Write your code with `/data` persistence in mind * Test your code locally if possible **2. Deployment and setup**: * Deploy your instance with the right template * SSH or access Jupyter immediately * Run your initialization script if needed (install temporary packages) * Download datasets to `/data` if they're on external storage **3. Active work**: * Run your training or inference workloads * Save all outputs, checkpoints, and logs to `/data` * Monitor progress through Jupyter or SSH * Use tmux/screen for long-running processes **4. Completion and cleanup**: * Verify all important data is in `/data` * Upload final results to external storage * Download any needed files to your local machine * Delete the instance to stop billing **5. Resumption (if needed)**: * Deploy a new instance with the same template * Run initialization script again * Download data from `/data` or external storage * Resume work from last checkpoint
Can I use GPU AI Service for production applications? Yes, GPU AI Service is designed to be used for production applications: **Suitable production use cases**: * **Inference serving**: Deploy models for prediction APIs * **Batch processing**: Regular scheduled model inference on datasets * **Model training pipelines**: Automated retraining workflows **Important considerations for production**: * **Stateless design**: Your application should not rely on local state outside `/data` * **External storage**: Use S3, databases, or cloud storage for persistent data * **Monitoring**: Implement logging and monitoring for your application * **Restart tolerance**: Design your application to handle instance restarts gracefully * **Reserved instances**: Use reserved pricing for cost-effective 24/7 operation **Not suitable for production**: * Applications requiring guaranteed uptime with no restarts * Complex multi-service architectures requiring container orchestration * Applications needing persistent custom system configurations
*** ## Templates and Customization
What's the difference between using a pre-configured template vs. plain Ubuntu? **Pre-configured templates (recommended)**: Advantages: * Immediate productivity: Start working in seconds * Optimized configurations: Framework-specific optimizations pre-applied * GPU-ready: CUDA, drivers, and libraries all compatible * Best practices: Environments configured following industry standards * Time-saving: No manual installation or troubleshooting Disadvantages: * Less flexibility: You get what's in the template * May include unused packages: Templates have broader tool sets **Plain Ubuntu template (not recommended for most users)**: Advantages: * Complete control: Install exactly what you need * Minimal base: Start from a clean slate Disadvantages: * Time-consuming: Manual installation of all dependencies * Compatibility risks: You might install incompatible versions * Not persistent: All installations lost on restart, requiring repeated setup * More complex: Requires deep knowledge of CUDA, drivers, framework installation **Recommendation**: Use pre-configured templates unless you have a very specific reason not to. The time saved and stability gained far outweigh any perceived advantages of plain Ubuntu.
What is the "Bring Your Own AI Template" (BYO) feature? The BYO AI Template feature allows you to create and use custom templates that persist across restarts. This solves the limitation of the plain Ubuntu template. **How BYO AI Template will work**: 1. Start with a base template or plain Ubuntu 2. Install your custom packages and configurations 3. Build image and push to any OCI-compliant container registry 4. Deploy future instances using your custom template **Benefits**: * Custom packages available immediately on deployment * No need to reinstall packages after every restart * Share templates with your team * Maintain consistent environments across projects **When to use BYO AI Template**: * You need specific package versions not available in standard templates * Your workflow requires unique combinations of tools * You want to standardize your team's development environment * You frequently work on similar projects with specific requirements
How do I request a new pre-configured template? If many users could benefit from a specific template configuration: 1. **Check existing templates**: Ensure your needs aren't already met 2. **Document your requirements**: List the specific packages, versions, and configurations needed 3. **Explain the use case**: Describe what workloads this template would support 4. **Contact support**: Submit a template request through your dashboard or support channels NeevCloud regularly adds new templates based on user demand and emerging AI/ML trends.
*** ## Troubleshooting and Common Issues
My installed packages disappeared after restarting. Why? This is expected behavior. All software installed outside the `/data` directory is lost when your instance restarts. Only data in `/data` persists. **Solution**: * Use pre-configured templates that include your needed packages * Create an initialization script in `/data` to quickly reinstall packages after restart
I can't run Docker inside my instance. What's wrong? Docker and nested containers are not supported in GPU AI Service instances. The instance itself is already running inside a container. **Solution**: * Adapt your workflow to run without Docker * Use Python virtual environments instead of containers for isolation * If you absolutely need Docker, consider alternative NeevCloud services
My training job stopped after I closed my SSH connection. How do I keep it running? When you close your SSH connection, any processes you started in that session are terminated unless you use a terminal multiplexer. **Solution**: Use tmux or screen to maintain persistent sessions: ```bash # Start a tmux session tmux new -s training # Run your training python train.py # Detach from tmux: Press Ctrl+b, then d # Your training continues running # Later, reattach to see progress tmux attach -t training ```
I saved my model checkpoint but it's gone after restart. Where is it? If you saved your checkpoint outside the `/data` directory, it was lost during restart. **Solution**: Always save important files to `/data`: ```python # Wrong - saved to /workspace (not persistent) torch.save(model.state_dict(), "/workspace/model.pth") # Correct - saved to /data (persistent) torch.save(model.state_dict(), "/data/models/model.pth") ```
My instance keeps restarting. What's happening? Instances restart automatically if the main container process exits or crashes. **Common causes**: * Your training script encountered an error and exited * You ran a command that killed the main process * Out of memory errors causing crashes * Infrastructure issues (rare) **Solution**: * Check logs to identify why your script is exiting * Wrap your code in try-except blocks to handle errors gracefully * Use tmux to keep processes running independently of SSH * Monitor GPU memory usage to prevent OOM errors * Contact support if restarts persist without obvious cause
How do I know how much storage I have in `/data`? Check storage usage with: ```bash df -h /data ``` This shows total capacity, used space, and available space in `/data`.
Can I access my instance from multiple locations simultaneously? Yes, you can connect from multiple SSH sessions or have multiple browser tabs open to Jupyter simultaneously. However, be careful about: * **File conflicts**: Multiple processes writing to the same file can cause corruption * **Resource contention**: Multiple training jobs competing for GPU resources * **Session management**: Keep track of which sessions are running what processes
*** ## Getting Started Recommendations
I'm new to GPU AI Service. Where should I start? Follow this beginner workflow: **Step 1: Choose the right template** * For PyTorch projects: Select "PyTorch + CUDA" template * For TensorFlow projects: Select "TensorFlow + CUDA" template * For Stable Diffusion: Select "Stable Diffusion + ComfyUI" template * For LLM fine-tuning: Select "PyTorch + Transformers" template **Step 2: Deploy a small instance first** * Start with a single GPU instance * Use on-demand pricing while learning * Choose lower VRAM (16-24 GB) to minimize costs **Step 3: Test your workflow** * Upload a small dataset to `/data` * Run a short training job (few epochs) * Verify outputs are saved to `/data` * Practice stopping and restarting your instance **Step 4: Scale up** * Once comfortable, move to larger GPUs * Consider reserved pricing for extended work * Add more GPUs if needed (up to 8)
What should I do immediately after deploying my first instance? **First 5 minutes**: 1. **Test SSH access**: ```bash ssh -i ~/.ssh/your-key root@your-instance-ip ``` 2. **Verify GPU access**: ```bash nvidia-smi ``` 3. **Check your template environment**: ```bash python --version pip list ``` 4. **Explore the `/data` directory**: ```bash ls -la /data mkdir -p /data/datasets /data/models /data/outputs ``` 5. **Test a simple GPU operation**: ```python import torch print(torch.cuda.is_available()) print(torch.cuda.get_device_name(0)) ```
What are the most common mistakes to avoid? **Mistake 1: Saving important data outside `/data`** * Always use `/data` for datasets, models, and results * Check file paths in your scripts before long training runs **Mistake 2: Choosing plain Ubuntu template unnecessarily** * Use specialized templates that match your framework * Avoid manual installation unless absolutely necessary **Mistake 3: Not using tmux/screen for long jobs** * Always use terminal multiplexers for training that takes hours * This prevents job termination when SSH disconnects **Mistake 4: Running Docker or nested containers** * Adapt your workflow to run directly in the instance * Don't try to install Docker or container runtimes **Mistake 5: Forgetting to stop instances** * Set up cost alerts in your dashboard * Automate shutdown after job completion when possible **Mistake 6: Not creating checkpoints during training** * Save checkpoints regularly to `/data` * Resume training from checkpoints after interruptions **Mistake 7: Over-provisioning resources** * Start with smaller instances and scale up if needed * Profile your actual GPU usage before requesting 8 GPUs
*** ## Making the Right Choice
Is GPU AI Service right for my project? **GPU AI Service is an excellent fit if you**: * Need GPU-accelerated AI/ML workloads * Want immediate access without infrastructure setup * Work with standard frameworks (PyTorch, TensorFlow) * Can design stateless or `/data`-persistent workflows * Need flexibility to scale GPU resources on demand **Consider alternatives if you**: * Need persistent custom system configurations * Require Docker or container orchestration * Have non-AI/ML computational requirements * Need complete control over the operating system
What questions should I ask before deploying? Before deploying your first instance, ask yourself: 1. **Does a pre-configured template match my framework?** * If yes: Use that template and save time * If no: Create your own AI Template or request a new template 2. **Can I store all important data in `/data`?** * If yes: GPU AI Service will work well * If no: Reconsider your workflow or use external storage 3. **Do I need Docker or nested containers?** * If yes: GPU AI Service is not suitable * If no: Proceed with deployment 4. **How long will my workload run?** * Short (hours): Use on-demand pricing * Long (days/weeks): Consider reserved pricing 5. **How many GPUs do I actually need?** * Start small (1-2 GPUs) and scale up if needed * Profile your workload before committing to 8 GPUs
Where can I get help if I'm unsure? * **Documentation**: Review the complete NeevCloud technical documentation * **Dashboard**: Check template descriptions and instance specifications * **Support**: Contact NeevCloud support with specific questions about your use case * **Community**: Join NeevCloud discord server for user experiences and advice * **Trial**: Start with a small on-demand instance to test your workflow before committing
*** ## Quick Decision Guide Use this quick reference to determine if GPU AI Service fits your needs: | Your Requirement | GPU AI Service Suitable? | Recommendation | | --------------------------------- | ------------------------ | ---------------------------------------------- | | Training deep learning models | ✅ Yes | Perfect use case | | Fine-tuning LLMs | ✅ Yes | Use PyTorch + Transformers template | | Running inference | ✅ Yes | Consider smaller GPU for cost savings | | Jupyter notebook development | ✅ Yes | Use templates with JupyterLab | | Need custom packages | ⚠️ Partially | Use BYO template (coming soon) or init scripts | | Running Docker containers | ❌ No | Use alternative NeevCloud services | | Need persistent installations | ❌ No | Wait for BYO AI Template feature | | Data outside `/data` must persist | ❌ No | Only `/data` is persistent | | Need more than 8 GPUs | ❌ No | Deploy multiple instances or contact sales | ✅ = Fully supported and recommended\ ⚠️ = Possible with specific considerations\ ❌ = Not supported or not recommended