GKE Is Built on Google’s Kubernetes Expertise – So Why is it Still Underutilized?

Google didn’t just adopt Kubernetes. Google created it.

More importantly, Google has been able to dogfood Kubernetes at enormous scale for years, giving GKE the benefit of real-world experience from one of the largest Kubernetes fleets in the world. That experience has shaped the platform and led to capabilities such as Autopilot, which automate parts of the infrastructure management that become increasingly difficult to handle manually at fleet scale.

But that doesn’t mean the utilization problem has disappeared.

The 2026 State of Kubernetes Optimization Report found average CPU utilization on GKE of just 8%, memory utilization of 20%, and GPU utilization of 6%. CPU overprovisioning reached 69%, while memory overprovisioning reached 79%.

The report is based on direct measurements from more than 23,000 Kubernetes clusters and millions of compute resources across AWS, GCP, and Azure, collected before automated optimization was enabled.

This isn’t a story about GKE failing to solve Kubernetes infrastructure. Google can build the scheduler, automate node provisioning, and abstract away much of the infrastructure. What it cannot do is know whether a workload requesting four CPUs actually needs four CPUs.

That part still belongs to the people running the workloads.

The Hard Part Is Knowing Where to Look

The first mistake is to treat low utilization as a node problem.

On GKE, the same workload can create waste in different ways depending on how the cluster is running. In Autopilot, an oversized resource request directly increases what you pay for the workload. In Standard, that request can also affect scheduling and cause the cluster to provision more node capacity than the application actually consumes.

That makes resource requests one of the highest-leverage places to start. The report found that 69% of GKE clusters overprovision CPU, up from 40% the previous year, while 79% overprovision memory.

And there is an important distinction between being conservative and being oversized. A workload needs enough headroom to handle real production variability. The problem is when a number chosen defensively becomes the permanent number, long after anyone remembers why it was chosen.

A better baseline comes from actual workload behavior. Look at CPU and memory consumption over a representative period that captures normal traffic patterns, then compare that with the requests currently declared in the workload. For production workloads with weekly traffic cycles, a 7–14 day window gives a much more useful picture than a snapshot taken during a quiet hour.

This is also where GKE’s two operating models become practically relevant. In Autopilot, rightsizing requests is directly a cost optimization exercise. In Standard, it can reduce the amount of node capacity the cluster needs in addition to reducing the workload’s requested footprint.

In Practice: Don’t Fix a Cluster-Wide Problem With a Cluster-Wide Change

Suppose a production service requests 2 CPUs and 2 GiB of memory, but its normal usage is closer to 500m CPU and 700 MiB of memory. Increasing or decreasing the node pool won’t fix that mismatch. The workload is telling Kubernetes it needs resources that it rarely consumes.

The safer approach is to start with a small group of workloads, establish what they actually use, and adjust their requests based on observed behavior. If those changes behave as expected, expand the scope rather than changing hundreds of deployments at once.

That is particularly important when HPA is involved. VPA can continuously adjust resource requests, but running automated VPA against an HPA that also scales on CPU or memory can create a feedback loop: VPA changes the request, which changes the HPA calculation, which changes the number of pods, which changes utilization, which feeds back into VPA.

The practical approach is to start VPA in recommendation-only mode, or use application-level HPA signals such as request rate or queue depth when both are needed.

There is also evidence that rightsizing doesn’t have to mean trading reliability for cost. In the report, automated rightsizing cut provisioned CPU by roughly half while reducing OOM kills to near zero.

The point isn’t that every workload should immediately receive smaller requests. It’s that production data can tell you where the padding is actually helping and where it isn’t.

Once the Workload Is Right-Sized, the Nodes Have a Different Job

Rightsizing pods is only half of the problem on GKE Standard. The next question is how efficiently those workloads are placed on the underlying Compute Engine fleet.

Cluster Autoscaler remains the core autoscaling mechanism for GKE Standard, and Node Auto Provisioning can extend it by creating node pools when workloads need machine types that aren’t already available in the cluster.

The important point is that autoscaling cannot compensate for inflated workload requirements. If ten services each request twice the CPU they normally consume, the autoscaler has no way to know those requests are excessive. It will provision capacity to satisfy them.

Once requests are accurate, however, autoscaling and node consolidation have a much better baseline to work with. Nodes can be packed more efficiently, smaller or more appropriate machine types become viable, and the cluster can remove capacity that is genuinely no longer required.

That is where GKE Standard gives practitioners more control than Autopilot, but also more responsibility.

The trade-off is straightforward: if you own the nodes, you also own the unused capacity.

Spot Makes the Remaining Capacity Cheaper

Once the baseline is right-sized, the next question isn’t necessarily whether all of that capacity needs to be on-demand.

GKE Spot VMs can provide substantial savings for workloads that can tolerate interruption. Google’s Spot model provides roughly 30 seconds of preemption notice, so this isn’t a capacity type you simply turn on for everything.

The appropriate candidates are familiar: batch workloads, CI/CD runners, stateless services with enough replicas to tolerate node loss, and ML training workloads that checkpoint their state.

The production configuration matters as much as the discount. Workloads need appropriate disruption budgets and termination handling, and the cluster needs an on-demand fallback for cases where Spot capacity disappears. Without that fallback, an optimization intended to reduce compute cost can become an availability problem.

The useful point is that Spot can be part of the same capacity strategy rather than a separate optimization exercise. Keep workloads that require predictable availability on reliable capacity and let interruption-tolerant workloads consume cheaper capacity when it is available.

GKE Is Costing You an ARM and a GPU

Not all compute waste looks the same.

CPU and memory are often a rightsizing problem; GPUs and CPU architecture are more about getting the right amount of expensive hardware for the workload.

The report found average GPU utilization on GKE of just 6%, with substantial variation throughout the day, while GPU sharing remained relatively uncommon. If an inference workload only needs a fraction of a GPU most of the time, giving it exclusive access to an entire GPU may be simple, but it isn’t necessarily efficient. Depending on the GPU and workload, sharing or partitioning can let multiple workloads make better use of the same physical hardware.

The report includes a production example where seven models were moved to GPU time-slicing alongside better bin-packing and Spot usage, reducing costs by more than 70% compared with the previous platform while maintaining latency. The point isn’t to make the utilization percentage look better. It’s to get more useful work out of expensive hardware before adding more of it.

The same principle applies at the CPU architecture level. ARM now represents about 9% of the Kubernetes CPU fleet and has been growing 3.5 times faster than x86 since Q2 2024. GKE offers several ARM options, including T2A and Google’s Axion-based C4A and N4A families, giving teams another way to improve the economics of suitable workloads.

This doesn’t mean migrating an entire cluster. For scale-out workloads that can run on linux/arm64, move a small set of workloads to ARM, compare performance and cost, and expand from there. The important part is to make the workload efficient first. Moving an incorrectly sized workload onto a cheaper processor doesn’t solve the utilization problem; it just makes the wrong amount of compute slightly cheaper.

Commit to What You Actually Need – GKE Optimization is Possible

Only after the workload and infrastructure footprint are under control does it make sense to think about commitments. GCP’s Sustained Use Discounts apply automatically to eligible Standard node usage, while Committed Use Discounts are a deliberate commitment against a baseline. Committing before understanding that baseline can turn today’s overprovisioning into a multi-year financial commitment.

The objective isn’t to minimize the number on a single invoice line. It’s to establish a baseline that reflects the capacity workloads consistently need, separate that from burst capacity that can use Spot, and then use GCP’s pricing mechanisms to make the remaining baseline cheaper.

The interesting thing about GKE isn’t that Google has somehow eliminated Kubernetes waste. It’s that Google has already solved many of the infrastructure problems that become difficult at enormous scale, while the data shows that utilization still requires practitioners to make the right decisions about their workloads and infrastructure.

And the optimization path is fairly concrete: make workload requests reflect production behavior, make the underlying fleet efficient, move interruption-tolerant workloads to Spot, improve utilization of expensive GPU and ARM capacity where it makes sense, and commit only to the baseline that remains.

The key is not to optimize everything at once. The DevOps mindset applies here too: start with a small, controlled change, make sure it behaves as expected in production, and then roll it out more broadly. That’s how you improve utilization without turning cost optimization into a risky fleet-wide exercise.