Killing Clunky Kubernetes, ScaleOps Serves Marching Orders On ‘Unevictable’ Workloads
Kubernetes epitomizes movement. By its very nature, the cloud-native container orchestration technology is composed of a score (orchestral pun intended) of moving parts across a shifting topography of computing instances, dynamic application services and eminently flowing real-time data repository resources.
But some Kubernetes gets clunky.
Why? Because constraints set by developers cause certain workloads in Kubernetes clusters to become permanently fixed to specific nodes – a problem known as ‘unevictable’ instances. As a result, organizations are forced to maintain excess capacity, increasing cloud costs that traditional optimization tools cannot resolve.
Pod Placement, Por Favor
New York and Tel Aviv-based automated cloud resource management company ScaleOps thinks it has an answer. The company’s newly available Pod Placement feature reorganizes previously ‘unevictable’ workloads that traditionally couldn’t be moved or modified.
But why does this “phenomenon” happen and how has ScaleOps claimed to fix the predicament?
According to Guy Baron, co-founder and CTO of ScaleOps, modern Kubernetes production clusters often run a mix of workloads (from flexible workloads with evictable pods to workloads with pods that can not be touched) in live production. We can add to that mix with the fact that developers who own the workloads add their own scheduling and eviction constraints, such as Pod Disruption Budgets (PDBs) that define the maximum number of pods from a replicated application that can be simultaneously unavailable during voluntary disruptions like node maintenance or cluster upgrades.
Application Affinity Proclivity
This pod budgeting means that a minimum level of application availability can be detailed and guaranteed during disruption events. All of this comes down to a question of “affinity” (i.e. the propensity and proclivity for one part of the Kubernetes universe across pods, nodes and clusters to have a bonded relationship with another), not wholly dissimilar from the way we talk about application code dependencies. The existence of affinity and anti-affinity rules can have a cluster-wide impact and create serious challenges.
The result here is that even after rightsizing Kubernetes workloads, a significant amount of resources get strung up as they can not be reclaimed, causing a long-lasting impact and degradation of cluster utilization.
“What happens next is that unevictable workloads are ‘pinned’ to specific nodes once scheduled. This immutability prevents the cluster auto-scaler from efficiently bin-packing the cluster,” explained Baron. “The outcome? Underutilized nodes and wasted resources, as these pinned pods restrict the autoscaler’s ability to consolidate nodes and optimize resource utilization. To complicate matters further, these unevictable pods typically have a long lifespan.”
False “safe-to-evict: false”
Other unevictable problem pods can surface in situations where the pod in question is running critical system services and it carries a “safe-to-evict: false” annotation. This prevents the pod from being evicted during cluster operations, limiting the autoscaler’s ability to move workloads efficiently. Where Kubernetes has allowed the marking of pods with a safe-to-evict: false annotation, it ensures they remain on a node.
There’s also naked pods. This is where pods are created directly, without being managed by a higher-level controller, such as a Deployment, ReplicaSet, or StatefulSet. The Cluster Autoscaler does not evict naked pods, so a node running a naked pod will not be scaled down and “bin-packed” (as mentioned above). For completeness here (as network engineering and developers will know) bin packing (as it is in real life and as it is in computer science) involves filling up an optimally minimized number of container bins with a finite capacity for any given environment.
Advanced Scheduling Algorithms
ScaleOps’ Pod Placement technology introduces a new approach to this challenge through advanced scheduling algorithms that group unevictable workloads on a common set of nodes. This new capability automatically gains insight into the workload’s application-specific context, examining workload requirements and system constraints to optimize placement while maintaining critical service safeguards.
Where traditional approaches in this space require complex configuration changes, the company here says that Pod Placement works alongside existing Kubernetes schedulers and cluster auto-scalers, delivering additional cost savings.
“After deploying to over a thousand production clusters, the new Pod Placement capability automatically identified that more than 40% contained unevictable workloads creating systemic inefficiencies that traditional optimization tools simply couldn’t address,” said Baron. “In many cases, we found that strict pod disruption budgets were responsible for nearly 50% more resources than necessary because the cluster autoscaler couldn’t consolidate the nodes effectively. Pod Placement is the first solution to solve this at the scheduling layer, working in harmony with existing Kubernetes components.”
Baron and team claim that early adopters have seen their node counts “drop significantly” while maintaining strict availability requirements.
Zero-Touch, Just A Click
While AWS’ advice detailing unevictable pods across Kubernetes containers remains rather rawer (although arguably no less powerful) at the time of writing, ScaleOps has clearly sought to bring a new level of user convenience to bear. The platform’s zero-touch deployment requires no changes to existing workloads, pod specifications, or DevOps processes.
Baron and the ScaleOps team round out by saying that the company’s Pod Placement technology continuously and automatically optimizes workload distribution, while dashboards allow teams to monitor the ongoing cost savings and efficiency gains in real-time.
