vCluster Adds Virtual Kubernetes Reference Architecture for GPUs
vCluster Labs has made available a reference architecture for incorporating graphical processor units (GPUs) running artificial intelligence (AI) workloads into virtual Kubernetes clusters.
Company CEO Lukas Gentele said the Infrastructure Tenancy Platform for AI reference architecture will make it simpler for IT teams to deploy AI inference workloads on platforms based on Kubernetes clusters, including NVIDIA DGX systems and the cluster management software for high-performance computing (HPC), Kubernetes, and OpenStack environments that NVIDIA gained with the acquisition of Bright Computing.
The Infrastructure Tenancy Platform for AI reference architecture includes instances of vCluster Private Nodes, vCluster VPN based on Tailscale network virtualization software and a vCluster Auto Nodes capability along with integrations with NVIDIA Base Command Manager, KubeVirt software for encapsulating virtual machines and the network isolation controller developed by Netris.
There is also a secure container sandbox that can be used to isolate AI workloads natively on a Kubernetes cluster without having to employ a virtual machine.
The overall goal is to make it easier for IT teams to optimally consume scarce GPU resources that can be more easily shared across multiple AI workloads running in isolated virtual clusters, said Gentele. That approach also makes it possible to spin up virtual clusters based on GPUs in seconds, a process that otherwise can take the better part of an hour, he noted.
Interest in maximizing utilization of GPUs is, of course, running high given the scarcity issues that organizations building AI applications are encountering. As such, an ability to spin up virtual clusters in a way that increases utilization of GPUs provides a significant incentive for organizations that are building and deploying AI to standardize on Kubernetes, noted Gentele.
At a time when more organizations than ever are concerned about the total cost of IT, vCluster software provides a means for rapidly spinning up virtual clusters using a shared set of infrastructure in a way that ultimately improves utilization, added Gentele.
Those virtual clusters can then be managed by DevOps or platform engineering teams using command line interfaces (CLIs), custom resource definitions (CRDs), Helm charts, infrastructure-as-code (IaC) tools and YAML files, or by IT administrators using a graphical user interface. The latter approach expands the available pool of IT talent capable of managing Kubernetes clusters. Many Kubernetes clusters are initially deployed by DevOps engineers who have experience working with YAML files, but as the number of clusters increases, the need to enable IT administrators who have less programming expertise has become more pressing.
Virtual Kubernetes clusters today are used most widely in pre-production environments to reduce the total number of physical Kubernetes servers an organization needs to deploy. However, the number of instances of virtual clusters being used in production environments is growing as more organizations look to reduce the total cost of IT.
It’s not clear just how many virtual Kubernetes clusters may have been spun up in recent years, but given the potential number of virtual clusters that could run on a single physical cluster configured with GPUs and CPUs, it may not be too long before those virtual clusters far outnumber existing physical clusters, with the number of virtual nodes running to also soon exponentially increase.
