Kubernetes v1.35 Arrived, Right On Workload-Aware Schedule
The workload is the application. Although that might sound like a throwback to the Sun Microsystems tagline of old (the network is the computer), the parallel is conveniently close if we consider the fact that today we specify a “workload” exists as an application running on Kubernetes.
Whether a workload takes the form of a single component, a couple of components or several working in concert, as we know, a Kubernetes workload runs inside a set of pods, where a pod itself represents a set of containers running in a cluster, for a defined lifecycle.
Workloads are, of course, essential, largely because they help us to deploy, scale and manage applications running Kubernetes. They also help us to define the internal mechanics of a Kubernetes cluster so that we can define how pods and containers are configured, created, exposed and maintained.
In practice, running Kubernetes requires a high degree of time management. Not necessarily from developers themselves (although actually, yes, that too), but from the point of view of scheduling. In live production, scheduling large workloads is a much more complex and fragile operation than scheduling a single pod, as this process often requires a consideration of all pods together, instead of scheduling each one independently.
Knowing Your Network Nuances
Google software engineers Maciej Skoczeń and Dominik Marciński explain where workload scheduling nuances are today.
Some workloads (they cite a machine learning batch job) are more efficiently executed if compute elements are placed on the same server rack. The pods that form that workload might have identical scheduling requirements, but not necessarily; therefore, knowing what process and schedule the whole system is defined by is clearly fundamental.
The pair say that, considering how common and important workload scheduling is to Kubernetes, it’s now “high time” to make workloads a first-class citizen for kube-scheduler (the Kubernetes control plane component that acts as a pod placement brain) and support them natively.
“The recent 1.35 release of Kubernetes delivered the first tranche of workload-aware scheduling improvements. These are part of a wider effort aiming to improve the scheduling and management of workloads. The effort will span over many special interest groups (SIGs) and releases. It is supposed to gradually expand the capabilities of the system toward reaching its North Star [goal], which is seamless workload scheduling and management in Kubernetes, including, but not limited to, preemption and autoscaling,” write Skoczeń and Marciński on the Kubernetes blog.
The Google duo highlight the fact that Kubernetes v1.35 introduced the Workload API resource. This enables cloud-native software engineers to describe the “desired shape” as well as scheduling-oriented requirements of a workload.
Wanna be in my Gang (Schedule)?
It comes with an initial implementation of “gang scheduling” that instructs the kube-scheduler to schedule gang pods. As the K8S cognoscenti will know, gang scheduling gang pods means pods (in a defined gang) are all scheduled together and run in what is known as an “all-or-nothing” method to ensure all the component parts of an application start simultaneously. This feature, along with the new Workload API feature are released in Alpha for v1.35 – it is disabled by default and will continue to mature in future release cycles.
Without gang scheduling, a job might be partially scheduled, consuming resources without being able to run, leading to resource wastage and potential deadlocks.
The engineers behind this work point out that while this is a first implementation, the Kubernetes project firmly intends to “improve and expand” the gang scheduling algorithm in future releases. Expected (or hoped for) benefits include a single-cycle scheduling phase for a whole gang, workload-level preemption, and more. As an additional note here on gang scheduling, it’s worth clarifying any misunderstanding that while pods will now be scheduled all-or-nothing as a group, they are not necessarily guaranteed to “start simultaneously” at the same moment; although, since they are scheduled altogether, it will be close.
Opportunistic Batching, Performance Optimization
Additionally, the team says that they have improved the scheduling of identical pods (that typically make a gang) to speed up the process. This is thanks to the opportunistic batching feature (currently in beta release), a performance optimization feature that groups and schedules similar workloads (such as identical pods) together by reusing scheduling calculations.
“This is a Beta feature that improves scheduling latency for identical pods,” explain Skoczeń and Marciński. “Unlike gang scheduling, this feature does not require the Workload API or any explicit opt-in on the user’s part. It works opportunistically within the scheduler by identifying pods that have identical scheduling requirements (container images, resource requests, affinities, etc.). When the scheduler processes a pod, it can reuse the feasibility calculations for subsequent identical pods in the queue, significantly speeding up the process.”
The North Star Vision
This arm of the total Kubernetes team base defines its ambitions around what has been a North Star vision for efficiency and effectiveness. The team explains that the project has a broad ambition to deliver workload-aware scheduling in its most usable and functional form. The new APIs and scheduling enhancements detailed here are said to be just the first steps.
Plans for the project include introducing a formalized workload scheduling phase, creating improved integration between scheduling and autoscaling, introducing multi-workload scheduling simulations, and delivering enhanced support for multi-node Dynamic Resource Allocation (DRA) and topology-aware scheduling.
