Skip to content

NVIDIA GPU Operator on Kubernetes: The Complete Guide

From the basics to advanced setup, performance tuning, and getting the most out of your GPUs

Version note: The GPU Operator, its components, and Kubernetes change quickly. The commands and values here follow current GPU Operator conventions (v24.x/v25.x). Before you run anything in production, check the exact chart version, the driver branch, and the support matrix in the official docs.

Contents

  1. Fundamentals: GPUs in Kubernetes
  2. What the GPU Operator Is and Why You Need It
  3. Architecture and Components
  4. Prerequisites and Planning
  5. Installation (Basic)
  6. Installation Scenarios (Intermediate)
  7. Verifying the Installation
  8. Scheduling GPU Workloads
  9. GPU Sharing: Time-Slicing, MPS, MIG, vGPU
  10. Dynamic Resource Allocation (DRA) for GPUs
  11. Monitoring and Observability (DCGM)
  12. Performance Optimization: Node and Hardware Level
  13. Performance Optimization: Kubernetes Level
  14. Performance Optimization: Networking and Multi-Node (RDMA, NCCL)
  15. Performance Optimization: Storage and Data Pipelines
  16. Performance Optimization: Application Level
  17. Using GPUs Efficiently: Utilization, Quotas, Autoscaling, Cost
  18. Day-2 Operations: Upgrades, Health, Reliability
  19. Security Considerations
  20. Troubleshooting Playbook
  21. Reference Architectures
  22. Cheat Sheet and Checklists
  23. Further Reading