Gitpod Leaves Kubernetes’ ‘Dead-End Failures’, Adopts Home Grown Tool
Enterprise open-source container application development organization Gitpod has stated its intention to “leave Kubernetes” for a variety of reasons. Describing its use of Kubernetes’ cloud-native container orchestration tools and services as a journey of “experiments, failures and dead-ends”, the engineering team will take its automated and standardized development platform forward to a home-grown technology known as Flex.
Seemed Obvious, at First
After a series of coding headaches brought on by what the team calls out as complexity issues, resource management challenges and state management problems, Gitpod co-founder and CTO Christian Weichel along with staff engineer Alejandro de Brito Fontes have said that they were initially drawn to Kubernetes because it “seemed like the obvious choice” for building out remote, standardized and automated development environments. As such, Gitpod has been using Kubernetes since the company’s inception in 2020.
“Over the years, we experimented with many [technologies] in pursuit of the most optimal infrastructure to balance security, performance and interoperability. All while wrestling with the unique challenges of building a system to scale up, remain secure [when] handling arbitrary code execution and be stable enough for developers to work in,” blogged Weichel and de Brito Fontes.
Initially drawn to key benefits of Kubernetes including its inherent scalability across container orchestration tasks and its widely popularized status which has created a rich ecosystem and community, Gitpod says that as its own technology and user base scaled, it encountered challenges around security and state management that pushed Kubernetes to its limits.
“Managing Kubernetes at scale is complex,” note Weichel and de Brito Fontes. “While managed services like GKE and EKS alleviate some pain points, they come with their own set of restrictions and limitations. We found that many teams looking to operate a cloud development environment underestimate the complexity of Kubernetes, which leads to significant support loads.”
Fundamentally, say the pair, Kubernetes is built to run well-controlled application workloads, not unruly development environments.
Forced Reverse Engineering
While Kubernetes’ complexity obstacles shouldn’t come as news to most software developers, the Gitpod team bemoans the amount of reverse engineering they claim to have endured while working to force development environments onto the Kubernetes platform. As such, the in-house team started building Gitpod Flex in January of 2024. It shipped 10 months later in October of this year.
Flex is founded on the Gitpod mantra which states that all software development needs to occur in secure, compartmentalized, interactive environments, while also being API-accessible and machine-comprehensible.
What’s Gitpod Flex like then? Well, foundationally, it’s a bit like Kubernetes.
“In Gitpod Flex we carried over the foundational aspects of Kubernetes such as the liberal application of control theory and the declarative APIs whilst simplifying the architecture and improving the security foundation,” wrote Weichel and de Brito Fontes. “[In Flex], we orchestrate development environments using a control plane heavily inspired by Kubernetes. We [also] introduced some necessary abstraction layers that are specific to development environments and cast aside much of the infrastructure complexity that we didn’t need – all whilst putting zero-trust security first.”
Through its runner-based architecture, Flex is designed to run on laptops, cloud instances and on-premises deployment zones. As many will know, runners are self-hosted software orchestrators that work to keep sensitive information secure within a private network while looking after operational functions including scaling, backup, caching and version control updates. Flex is claimed to keep source code, data and intellectual property within a software development team’s private network and, according to Gitpod, it’s “what developers have been asking for” in recent times.
Kubernetes is Incredible, Still
All that said and done then, the Gitpod team still has kind words to spare for K8s, its people and its ethos.
“Kubernetes is incredible. It’s supported by an engaged and passionate community, which builds a truly rich ecosystem. If you’re running application workloads, Kubernetes continues to be a fine choice. However, for system workloads like development environments [with a uniquely stateful nature], Kubernetes presents immense challenges in both security and operational overhead. Micro-VMs and clear resource budgets help, but make cost a more dominating factor,” say Weichel and de Brito Fontes.
Keen to pave a way forward for developers to be as productive as possible within secure boundaries and with low operational overhead, the Gitpod software engineering duo thinks that these requirements are possible with Kubernetes is possible, but these factors come at what they say is a significant cost. It is, in their terms, all about learning the difference between application and system workloads the hard way. Gitpod Flex can be deployed and self-hosted in less than three minutes and the team is already planning virtual demos as part of a promotional initiative.
