CAST AI Analysis Surfaces Massive Kubernetes Infrastructure Waste
An analysis of Kubernetes clusters deployed in cloud computing environments by more than 2,100 organizations finds the average CPU utilization across Kubernetes clusters is just 10%, with memory utilization rates standing at only 23%, according to a report published today by CAST AI.
Overall, the report finds the over provisioning gap between requested resources and the infrastructure used is at 40% for CPUs and 57% for memory.
CAST AI president Laurent Gil said that given the uniform level of inefficiency being achieved by IT teams that have deployed Kubernetes on cloud services from Amazon Web Services (AWS), Microsoft and Google, the platform itself remains challenging to optimize.
As a result, the total cost of deploying cloud-native applications is much higher in many organizations than necessary, he added.
None of the organizations analyzed were running any type of third-party Kubernetes management platform that would, for example, make it simpler to take advantage of spot pricing instances offered by cloud service providers to reduce costs. Clusters that partially leverage spot instances achieve an average compute cost reduction of 59%, while IT teams running clusters exclusively on spot instances can see even greater cost reduction of 77%, the report noted.
The report also notes that IT teams that opt to run workloads on Arm processors rather than x86 processors can save as much as 65% on infrastructure costs.
Additionally, the report notes that many Kubernetes clusters are now being used to run artificial intelligence (AI) workloads. Cast AI analyzed different regions and availability zones to determine where scarce graphical processor units (GPUs) needed to run those applications might be most easily found, with organizations that can move these workloads able to reduce costs by a factor of two to seven when using spot pricing and three to 10 when using on-demand services.
In general, application developers will over-provision infrastructure to ensure their applications remain available at peak loads. However, most of that infrastructure is never consumed. Each application deployed on a Kubernetes cluster then adds to the total amount of infrastructure deployed in the cloud that is being wasted, noted Gil.
In theory, IT teams are under more pressure than ever to optimize cloud costs, but historically there is a tendency among IT teams to simply accept high levels of infrastructure wastage as simply the way things have always been, he added. However, savings that could be generated by running workloads more efficiently in the cloud could clearly be applied by IT teams to other underfunded projects, noted Gil.
Hopefully, as more organizations embrace best FinOps practices to reduce cloud costs, the amount of IT infrastructure being wasted will steadily decline. In the meantime, IT teams might want to make a case for adopting platforms that will enable them to manage fleets of Kubernetes clusters in a way that will easily pay for itself, once their monthly cloud computing bills start to rapidly decline. Otherwise, all they are really doing is making cloud service providers a lot more profitable than they already are.
