Kubernetes and IPv6: Together at Last?
As administrators migrate their clusters from Ingress NGINX, which is being retired from the Kubernetes line-up next month, they might want to consider adding another task to their job: migrating their network to IPv6.
If they don’t have enough IPv4 addresses on hand, they may find difficulty in going through a side-by-side migration deployment to get everything over to the replacement Gateway API.
Not that IPv6 would be simpler to deploy, but it could be the better long-term solution.
This year at FOSDEM, Ole Mathias Heggem, a Norwegian DevOps consultant at Redpill Linpro, made a case that now is the time to consider using IPv6 on Kubernetes.
Heggem is an admitted IPv6 fanboy: He has used IPv6 since 2013, boasting of having ASN and a /29 block of IPv6 to run his home network.
With IPv6 “there is no need to extend the address space,” using NAT and other work-arounds, Heggem says.
With all the complexity inherent in a Kubernetes deployment, a simpler address space could be a relief for operators.
IPv6 Arrived But Did Not Conquer
By 2015, the year the Kubernetes cloud native container orchestrator was first pushed to GitHub , IPv6 was already old news.
The Internet Engineering Task Force (IETF) started work on IPv6 in the early 1990s. It was clear even then that the Internet would be running out of address space.
With its 32-bit address field, IPv4 can only accommodate 4,294,967,296 endpoints, which is, as Heggem points out, less than the current world population. Clearly, the Internet would exhaust all the IP addresses eventually. And it has, for the most part.
Introduced in 1998 through RFC 2460, IPv6 had a luxurious 128-bit address space, which provides for over 2 to the 128th power of addresses, or about 340 undecillion addresses, plus some change.
But, as IPv6 was being worked out, most Internet users in the mean time found that a simple hack solved the address shortage, namely Network Address Translation (NAT), which could hide a whole bunch of IP addresses behind one number.
Working with NAT can be messy, but most found an easier alternative than upgrading to IPv6. Today, many cloud providers use Carrier Grade NAT (CGNAT) to supply addresses for their accounts,
Kubernetes Users Were Slow to Adopt IPv6
So by the time Kubernetes came along, there was no compelling reason for its creators to worry about IPv6.
These days, however, clamor for IPv4 addresses is slowly growing. Google estimates that nearly 45% of its traffic is IPv6. This has been growing at a rate of 20% over the past five years, Heggem observes.
Since 2021, there has been a waiting list for IPv4 addresses, with 888 requests now in the queue, the top of which has been waiting for over 500 days.
And IPv4 is starting to show its limits. Service providers are starting to restrict usage to individual IPs. Docker Hub, for instance, limits pulls per IP number to 100 for every six hours. If you want to cadge an IP address from your Internet provider, it’ll probably cost you extra.
Then there’s the NGINX gateway retirement issue for Kubernetes. The Cloud Native Computing Foundation decreed that it would no longer be supporting the Ingress NGINX, the long-favored proxy server for connecting Kubernetes to the outside world. Instead, the user must move to the Gateway API-supporting controller, if they want to stay updated.
Moving a cluster to a new address space would best be done by setting up a duplicate environment, then switching over to the new controllers. But this assumes the operator has enough addresses, either of the IPv4 or NAT variety, to cover both the old and new network at once.
Kubernetes Current Support for IPv6
Kubernetes has supported the ability to run only on IPv6 since K8s 1.9 (December 2017). It supports IPv6 addressing for both pods and services.
The ability to run dual-stack IPv4 and IPv6 stacks has been supported since Kubernetes 1.23 (December 2021). In this case, pod and services can get both IPv4 and IPv6 addresses. Kubernetes services must be marked at single stack or dual stack.
At present, the managed Kubernetes offers from the three major cloud providers (AWS EKS, GCP GKE, Azure AKS) support dual-stack IPv4/IPv6, in varying levels of maturity, Hegge notes.
Why Kubernetes Should Move to IPv6
The chief reason that Kubernetes users should move away from IPv4/NAT implementations is that it will reduce the complexity of managing these setups.
Many use Dual stack setups, but that produces twice the debugging complexity.
There’s an inherent simplicity in an IPv6 world, where each pod gets its own IP address. No translation is needed to communicate with anything else in the world. Just pure end-to-end connectivity.
The operator will still have to think about security, if each pod gets a public address, though it is up to the operator if these will be exposed to the Internet. But no more NAT.
Another advantage is that you can have two clusters in the same IP space, with nodes easily able to communicate with one another, something that requires specialized multi-cluster software to do today.
Running multiple clusters in a single address space can be nimbler and offer more reliability than maintaining one giant cluster.
Challenges of IPv6
Of course, adopting IPv6 still puts users on the cutting edge, with all the pains that entail.
Many software libraries, cloud services and support tools do not yet offer full support of IPv6. GitHub, for instance, does not support IPv6 today.
The good news is that Cilium, Kubernetes’s favored underlay network, supports IPv6. In his presentation, Heggem went through how to install IPv6, using Helm charts, for the K9s distribution.
Communicating to IPv4-based hosts however would still require using something that NAT64 for the translation. Or a web proxy or a registry as a pull through cache.
But most of the work would be getting the rest of the users onboard with the very idea of IPv6.
IPv6 is still a scary thing for many people. IPv4 addresses are friendly, even memorable (“Home is 127.0.0.1”). IPv6’s hexadecimal notation is larger and more forbidding.
“Home is fc00::7” anyone?
