EKS Powers the Cloud’s Largest Kubernetes Fleets – So Why Is It Still Running Mostly Idle?
Amazon Elastic Kubernetes Service (EKS) remains the primary foundation for enterprise production Kubernetes, hosting massive mission-critical workloads across the globe. AWS pioneered modern cloud infrastructure twenty years ago, and today EKS provides the deepest catalog of compute options anywhere: thousands of EC2 instance types, Graviton silicon, Karpenter orchestration, Spot markets, and specialized AI accelerators.
Yet having the most mature cloud ecosystem has not insulated AWS environments from the industry-wide Kubernetes utilization crisis.
The 2026 State of Kubernetes Optimization Report makes that reality clear.
Across tens of thousands of clusters and millions of compute resources analyzed across AWS, Azure, and GCP, average CPU utilization fell from 10% to 8% in 2025. Memory utilization declined from 23% to 20%. Meanwhile, CPU overprovisioning jumped from 40% to 69%, and memory overprovisioning reached 79%.
Then there are GPUs. Across the report, average GPU utilization was just 5%.
On EKS, average GPU utilization matched that fleet-wide average of 5%. That means for every dollar spent on accelerated computing on AWS, 95 cents pays for compute cycles that do no useful work.
The report’s findings are drawn from direct measurements of production clusters before automated optimization was activated, capturing how these environments actually operate day to day.
This raises a critical question for AWS teams.
If EKS offers the most sophisticated autoscaling and compute portfolio on the market, why is infrastructure efficiency slipping as footprints expand?
The Karpenter Paradox: Why Advanced Node Autoscaling Isn’t Enough
Many engineering teams adopted Karpenter to move beyond the limitations of native Cluster Autoscaler and rigid EC2 Auto Scaling Groups. Karpenter is fast, flexible, and capable of provisioning right-sized EC2 instances based directly on pending pod specifications without requiring pre-baked node groups.
However, Karpenter, like any Kubernetes autoscaler, has an unavoidable operational constraint – it provisions capacity based on requested resources, not actual consumption.
When a team deploys a service requesting 4 vCPUs and 8 GiB of memory, Karpenter treats that request as a non-negotiable scheduling mandate. If that application actually consumes 500m of CPU and 1.5 GiB of memory under peak traffic, Karpenter still provisions an EC2 instance sized to satisfy the requested footprint.
This dynamic is why CPU overprovisioning reached 69%. Engineers defensively pad resource requests to avoid CPU throttling, degraded latency, or Out-of-Memory (OOM) evictions. Over-allocating rarely causes a production outage, so that defensive request becomes the permanent baseline, replicating across Helm charts and Terraform templates across environments.
The cluster infrastructure ends up perfectly configured according to the Kubernetes API, but badly out of sync with real application needs. When you manage your own EC2 nodes on EKS, you own and pay for all that unused headroom.
GPU Crunch on AWS – Capacity Blocks & Idle Silicon
Underutilizing general-purpose CPU and memory is costly, but underutilizing GPUs carries an entirely different financial penalty. An idle CPU core costs fractions of a cent; an idle GPU costs dollars per hour.
That cost burden grew heavier in January 2026 when AWS broke a two-decade precedent of steady compute price reductions by raising H200 Capacity Block prices by 15%, citing severe supply-and-demand imbalances. Yet despite capacity scarcity and rising prices, EKS GPU utilization remains stalled at 5%.
The primary cause is the traditional dedicated-allocation model. Most AI/ML serving workloads experience variable, spiky inference requests punctuated by extended idle windows. Granting an inference endpoint exclusive access to a dedicated GPU instance means paying full hourly rates while the physical hardware spends most of its time waiting.
Breaking that cycle requires moving to GPU time-slicing, dynamic sharing, and intelligent bin-packing.
In the report, edtech platform ALLEN Digital demonstrated this shift on AWS EKS. Operating seven machine learning models (including BGE-M3, LlamaGuard, and Multilingual E5 Large), they initially ran dedicated instances on Amazon SageMaker, where intermittent request loads left hardware mostly idle. By migrating to EKS and enabling GPU time-slicing alongside automated bin-packing and a 50/50 Spot/on-demand split, they consolidated multiple models onto shared physical GPUs. The result was an immediate reduction in total compute costs of more than 70% compared to SageMaker, with zero compromise on inference latency.
When GPUs are used efficiently, organizations do not need to hoard capacity or pay rising reservation premiums—they simply extract more useful work from fewer physical cards.
Spot On – Navigating Regional Survival Curves and 5x Price Arbitrage
EC2 Spot Instances offer up to 90% savings over on-demand rates, yet fewer than 2% of GPUs in the report were running on Spot. For production workloads, the concern over an abrupt two-minute interruption notice often keeps teams tethered to expensive on-demand instances.
The data shows that Spot interruption risk is not uniform—it is heavily determined by regional placement.
Tracking Spot survival curves for NVIDIA T4 instances (g4dn.xlarge) reveals stark differences between AWS regions:
- In stable regions like eu-west-3, the survival probability for a T4 Spot instance remains above 0.9 (90%) across a full 24-hour window.
- In congested regions like us-east-1 and eu-central-1, survival probability plunges below 0.2 within that same 24-hour timeframe, representing an 80% likelihood of interruption.
Pricing volatility follows an equally wide spread. For large GPU instances like p4d.24xlarge (A100), regional Spot pricing fluctuated by 2x to nearly 5x between US regions such as us-east-1, us-east-2, and us-west-2. Shifting workloads based on real-time market conditions delivered up to 80% cost reductions compared to the average Spot price.
Tracking Spot liquidity, regional interruption curves, and price spikes across availability zones manually is impractical for engineering teams. Automated Spot lifecycle management bridges that gap. For example, marketing platform Iterable utilized automated hourly Spot rebalancing across their AWS footprint to eliminate 2,000 unneeded CPUs, trim their node count by 10%, and generate a 75% reduction in compute spend while handling real-time traffic swings.
Graviton and the 93% Stateless Advantage
AWS set the pace for cloud silicon innovation with Graviton, and EKS users are adopting ARM faster than any other architecture. The report shows ARM CPU nodes growing 3.5 times faster than x86 since Q2 2024, now accounting for 9% of the total Kubernetes CPU fleet.
This trend directly matches the workload composition of modern clusters, where 93% of observed deployments are stateless.
Stateless microservices, background workers, and containerized APIs can easily run on linux/arm64 without persistent volume constraints or storage I/O limits.
When combined with automated rightsizing and aggressive bin-packing, Graviton delivers a compounding efficiency dividend. Migrating an oversized x86 pod to Graviton reduces per-core costs, but rightsizing the pod first and placing it onto Graviton eliminates both the unneeded allocation and the hardware premium at the same time.
Debunking the Headroom Myth
The most common justification for running overprovisioned EKS clusters is reliability: platform teams assume that large resource buffers protect service-level objectives (SLOs).
The report’s findings contradict this assumption.
Static overprovisioning provides a false sense of security. In one production environment studied, OOM kills consistently averaged 40 to 50 events per measurement window, occasionally spiking above 80, despite extensive resource padding across the cluster. Static buffers are applied broadly, leaving quiet services heavily padded while rapidly growing or memory-leaking workloads remain pinned to out-of-date limits until the kernel terminates them.
Automated workload rightsizing acts in both directions:
- It lowers CPU and memory allocations for workloads running below their requested footprints.
- It detects workloads experiencing memory pressure and increases their limits before evictions happen.
In practice, this dropped OOM kills to near zero while reducing the provisioned CPU footprint by approximately 50%. Teams do not have to trade reliability for cost efficiency; eliminating guesswork stabilizes the platform.
Rightsizing Before Committing to Protect AWS Savings Plans
AWS Savings Plans and Compute Reserved Instances are standard tools for curbing cloud spend, but committing to infrastructure before eliminating waste locks that waste into place.
If an EKS environment operates with 69% CPU overprovisioning, purchasing a three-year Savings Plan simply locks in a discounted rate for infrastructure that applications never use. When managed manually, drifting workloads and traffic shifts often cause organizations to waste up to 35% of prepaid commitment resources.
An effective optimization path on EKS follows a disciplined sequence:
- Rightsize workload requests based on observed production consumption cycles rather than initial estimates.
- Pack nodes efficiently using dynamic node provisioning and cost-effective instance families like Graviton.
- Route interruption-tolerant workloads to Spot with multi-AZ diversification and automatic on-demand fallback.
- Anchor commitments only to the true baseline that remains, keeping commitment utilization near 98%.
This is a good place to start, if you want a deeper dive on EKS cost optimization.
Across AKS, GKE, and EKS, the takeaway from the 2026 State of Kubernetes Optimization Report remains consistent. Low cluster utilization is not an issue with any specific cloud provider, it is an inherent operational challenge that arises whenever organizations scale Kubernetes manually.
As environments expand across multiple teams, regions, and instance types, no team can manually maintain every resource request, Spot price fluctuation, and node lifecycle decision. High utilization and rock-solid availability are not achieved through a one-off configuration tweak – they require continuous, automated feedback loops.
