CNCF Advances Longhorn Distributed Block Storage Platform
The Technical Oversight Committee (TOC) for the Cloud Native Computing Foundation (CNCF) has voted to advance the Longhorn distributed block storage system for Kubernetes to the next level.
Longhorn was originally developed by Rancher Labs and donated to the CNCF as a sandbox project. Rancher Labs was subsequently acquired by SUSE. Now an incubation-level project, the next step would be for Longhorn to graduate alongside Kubernetes.
Sheng Liang, president of engineering and innovation at SUSE and a member of the CNCF TOC, says that as more stateful applications are developed, the need for a modern block storage system that runs natively on Kubernetes is becoming more apparent.
Longhorn uses a control plane that is based on the controller design pattern. It leverages dynamic pod management to provision its data plane communication stack. The various components that make up the platform communicate via the gRPC protocol. It also provides support for incremental snapshots that can be used for backup to external platforms, non-disruptive upgrades that enable updates to the platform without taking nodes offline, an integrated dashboard and the ability to share metrics with the open source Prometheus monitoring platform that is also being advanced under the auspices of the CNCF.
The current Longhorn roadmap includes an enhanced application programming interface (API), cloud provider enhancements, highly available RWX storage and additional storage networking support. The most recent version of Longhorn added support for volume and backup encryption, automatic replica rebalancing, policy-based backup rules and volume clone support.
The number of contributors to Longhorn now stands at 800 from more than 120 companies and more than 34,000 nodes running an instance. Organizations that are already using Longhorn in a production environment include Cerner, Tribunal Regional Eleitoral do Pará and Tyk.
It’s not clear when Longhorn will officially graduate, but there is a clear race to enable stateful applications and make use of various types of persistent storage. Today, distributed block storage systems are most widely used in the cloud, but as IT environments become more hybrid, it’s only a matter of time before block storage platforms also are used in on-premises IT environments.
Kubernetes already creates permanent storage mechanisms for containers based on Kubernetes persistent volumes (PV). That makes it possible to access data far beyond the lifespan of any given pod. Kubernetes volumes allow users to mount storage units to expand how much data they can share between nodes. Regular volumes will still be deleted when the pod hosting that particular volume is shut down. The permanent volume, however, is hosted on its own pod to ensure data remains accessible. Upon creation, the PV is bound to the pod that requested the persistent volume claim (PVC).
Longhorn adds a distributed block system that makes it possible to deploy stateful applications at a larger scale across a distributed fleet of Kubernetes clusters. There are clearly going to be multiple approaches to bringing block storage to Kubernetes clusters, but, as is often the case, an open source approach could deliver a larger number of innovations faster.
