Beating Kubernetes Complexity With Instant Platform Engineering

Survey after survey shows that complexity, security and a skills gap are the top challenges organizations face in deploying Kubernetes.

In the latest CNCF Annual Survey, for example, the biggest challenges responders reported were lack of training and security. Lack of training was the top challenge cited by 44% of those who have yet to deploy containers in production and by 41% of those who use containers on a limited basis.

What these surveys are really telling us is that developers are overloaded. The combination of DevOps–which advocates for “you build it, you run it,” meaning each DevOps team is responsible for maintaining their own infrastructure–and the introduction of dozens of new technologies that make up a cloud-native stack, means that DevOps teams are now spending most of their time on Ops and not enough on Dev.

As more mission-critical workloads move to cloud-native platforms, this problem gets amplified. As workloads continue to grow in importance and cybersecurity threats become more prevalent, platforms need to provide robust service-level objectives (SLOs) for availability, resiliency and security. You no longer can get by with “good enough.” Platforms need to be designed and operated according to best practices.

Overcoming Kubernetes Complexity

With all these pressures, DevOps teams simply can’t keep up with the Ops demands. This is why platform engineering advocates for building a centrally managed developer platform shared by multiple teams instead of having each team build and run their own. That way, critical infrastructure tasks like security, governance and observability are done once and done right instead of being haphazard and duplicating efforts.

However, engineering a Kubernetes management platform is a complex task. A number of services must be assembled and integrated, including capabilities for fleet management, observability, networking, storage, security, disaster recovery, cost visibility, GitOps workflows, life cycle management and governance. Most in-house IT or DevOps teams do not have the engineering skills required to create a Kubernetes management platform, which is why so many do-it-yourself (DIY) efforts fail. Popular public cloud Kubernetes services were designed for the DevOps approach and provide limited centralized management capabilities for platform teams.

Instant Platform Engineering to the Rescue

Extending Kubernetes services with centralized management capabilities that enable a platform engineering approach across an organization’s entire fleet of clusters, in the cloud, on-premises and at the edge is what we call “instant platform engineering.” This deep integration and single-pane-of-glass capability is enabled by Cluster API (CAPI), the Kubernetes-native approach to cluster management via declarative APIs, which is one of the main reasons why Kubernetes is so robust and scalable. In contrast, other vendor solutions use scripts and legacy infrastructure-as-code (IaC) tools to do the same; these are brittle, require lots of maintenance and are prone to human error.

Platform teams want to align closely with the CNCF community about which components they integrate into their stack. Monolithic and proprietary “kitchen sink” platforms effectively cut them off from much of the innovation happening in the CNCF ecosystem. This is why many teams try (and often fail) to go the DIY route. Instant platform engineering, however, can alleviate the complexity of DIY routes and help developers drive innovation and value for their teams.

A Simple Way to Simplify DevOps

Instant platform engineering in the form of a production-ready Kubernetes management platform solves the complexity and skills gap challenges by providing a ready-made internal developer platform and “golden path” for DevOps teams, enabling them to devote their labor to creating business value rather than struggling to build a container management platform.

Rather than attempting to construct a Kubernetes platform from scratch or purchasing a platform that is too costly and difficult to get into production, organizations can purchase a Kubernetes platform that is easy to deploy and manage, is production-ready, is built on best-of-breed open-source components, integrates deeply with cloud Kubernetes services and is cost-effective.

Instant platform engineering also enables organizations to overcome the Kubernetes skills gap. Your workforce no longer has to grow as your infrastructure grows, and you have one way to manage your fleet instead of operational silos and duplicated efforts.

A Solid Foundation for the Future

Very few organizations today are at a maturity level that enables them to master multi-cluster and multi-cloud container management, let alone harness the more advanced capabilities that Kubernetes management platforms provide.

As organizations gradually mature in their Kubernetes capabilities, we will see more organizations reach higher maturity levels and deploy more advanced workloads, which will unleash a revolutionary wave of innovation. To get to those higher levels, organizations must first establish a solid Kubernetes foundation and instill the appropriate skills and culture within their management, infrastructure and DevOps teams.

Tobi Knaup

A cloud-native pioneer and evangelist, Tobi Knaup serves as the CEO of D2iQ. Previously, Tobi served as D2iQ’s Chief Technology Officer. As the primary author of the world’s first open source container orchestrator (Marathon) and co-creator of the KUDO toolkit for building Kubernetes Operators, Tobi has the unique ability to understand an organization’s cloud-native journey from all levels--business, technological and talent. And as the driver behind D2iQ’s next-generation Kubernetes platform, Tobi helps make it possible for organizations to navigate the cost and time-intensive challenges associated with enterprise-grade container orchestration. Before co-founding D2iQ, Tobi was one of the first engineers and technology lead at Airbnb, proving the technology’s value at scale in a production environment serving millions of users. A German native, Tobi holds a Bachelor of Science and a Master of Science from the Technical University of Munich.

Tobi Knaup has 3 posts and counting. See all posts by Tobi Knaup