AWS Stretches Elastic Kubernetes Service to Full Private Networking
While setting up AWS’s managed Kubernetes service, many security administrators probably realized that true Zero Trust security wouldn’t be possible if outbound K8s traffic still must flow over the AWS backbone to some public Internet-facing endpoint.
Now AWS has taken care of the issue.
Users of AWS’s Elastic Kubernetes Service (EKS) can now route Kubernetes outbound control plane traffic (“egress”) completely through their own virtual private clouds (VPCs). Surprisingly, this wasn’t possible before.
It’s good news for organizations that have adopted Zero Trust network isolation, in which all traffic must come through verified resources.
“Customers in regulated industries asked for a way to apply their own VPC egress controls to that path, so the policies that govern their workloads also govern the traffic that Kubernetes API Server initiates,” write a trio of AWS engineers in a blog post announcing the new EKS feature, Customer-routed control plane egress.
EKS is a managed service that spins up clusters under AWS’s own account. So any traffic emitting from these managed nodes – such as webhook callbacks, OpenID Connect (OIDC) provider lookups, aggregate API server requests – travels a public network to get back to the user’s VPC.
Now, a user can set a flag (“controlPlaneEgressMode = CUSTOMER_ROUTED” by way of the update-cluster-config command) and AWS will hand over that egress traffic to the user.
Warning: once the flag is set, it can never be reverted for the life of that cluster.
What Happens Next?
When the user enables customer-routed control plane egress on a cluster, EKS binds the Kubernetes API Server to the customer’s designated VPC Elastic Network Interface.
“VPC egress is your responsibility,” the engineers write.
From there, the customer must set the routes, security groups, and endpoints for the egress traffic rolling off the cluster. App calls may fail if the endpoint routes aren’t defined. DNS must be correct, and so on.
Also, note that service-side operations managed by the EKS control plane won’t be affected.
What about ingress traffic coming to the Kubernetes cluster? AWS solved the problem of managing inbound traffic several years ago through the Cluster Private Endpoint feature.
“Use both features together for a comprehensive private-networking posture. Private endpoint access secures inbound connectivity to the API server, while customer-routed control plane egress routes outbound customer-controllable traffic through your VPC,” the AWS engineers wrote.
Eliminating Public Endpoint for Private Data
Private egress will make security officers happy on a number of fronts.
For one, admission webhooks can be fully utilized. This Kubernetes extension validates each request made to a Kubernetes API server, and there’s always the suspicion that if the request travels over a public conduit, it could be forged somehow.
The same goes for using OpenID Connect to grant users access to the cluster. The handshake requires the control plane to contact the token issuer twice to secure authorization. Again, this is better done entirely within the VPC.
For different architectural reasons, public egress hasn’t been an issue with the other managed Kubernetes services from Microsoft Azure or the Google Cloud Platform. Azure embeds the Kubernetes API Server endpoints within a designated subnet within the user’s own VNet. Google doesn’t provide any public egress for clusters designed as private, offloading connectivity entirely to users.
