Nine Pillars of Kubernetes Best Practices

April 27, 2021April 26, 2021 Marc Hornbeek best practices, CI/CD, container orchestration, devops, kubernetes

In a prior blog, I explained Nine Pillars of Engineering DevOps with Kubernetes. The nine pillars represent categories of practices. In this blog, I expand on practices for each of the nine pillars. In a future blog, I will explain how to use these practices to conduct a gap assessment for your Kubernetes practices.

Leadership Practices for Kubernetes

Business leaders use Kubernetes to empower the organization to reach next-level performance, deliver innovative new software and features faster and enable multi-cloud operations for greater agility and resilience.
Leaders sponsor and encourage the use of Kubernetes within their teams.
Leaders ensure documentation and training cover the nine pillars of Kubernetes best practices described in this Kubernetes gap assessment.
Leaders monitor the use of Kubernetes to validate performance and ensure lessons learned are put into action.

Collaborative Culture Practices for Kubernetes

Kubernetes supports DevOps’ continuous learning culture by communicating deployment information with cross-functional teams.
Change management, often a source of conflict, is made more productive with Kubernetes because deployment configurations are version-managed together with the application code. This simplifies the ability of Dev, Ops and QA and security team members to collaboratively design end-to-end DevOps workflows.
Kubernetes enables standardized and centralized organizational workflows because it reduces the frustrations associated with scattered, decentralized information and disparate approaches to formatting and storing technical data.

Design-for-DevOps Practices for Kubernetes

Software is designed in accordance with the tenets of 12-factor apps, communicated through networked APIs, which work best for scalable deployments on clusters.
Kubernetes is used to orchestrate cloud-native applications. Modular distributed services are better able to scale and recover from failures.
Kubernetes versions are kept up to date.
Process whitelisting identifies unexpected running processes.

Continuous Integration Practices for Kubernetes

During the integration phase, container images are built and associated configuration, setup and run procedures are defined for Kubernetes.
Container images are kept small because smaller images can more quickly be orchestrated on a cluster.
The structure of container images is tested. Testing the structure ensures that all commands run as expected inside of your container. Testing also lets you check that specific files are in the correct location and have the correct content.
Built container images are scanned for vulnerabilities.
Container images are promoted and not rebuilt as they pass through the different stages of a CI/CD pipeline. Rebuilding can introduce differences across code branches.

Continuous Testing Practices for Kubernetes

Kubernetes facilitates enhanced end-to-end testing that more closely resembles the experience of users in production.
With Kubernetes, the work to stand up on-premises and cloud-based test environments is dramatically reduced, because containerized application software and associated testing software can be deployed in containers for testing as soon as application code is committed.
Developers use the tester’s Kubernetes instance for debugging. This eliminates long delays associated with developers and testers trying to replicate each other’s test environments. Kubernetes also helps testers and developers quickly exchange precise application configuration information.

Elastic Infrastructure Practices for Kubernetes

Infrastructure resources are clustered and can be consumed and released elastically, enabling seamless scaling and higher resource utilization.
Kubernetes eliminates many of the manual provisioning and other repetitive tasks of enterprise IT operations.
The unified and automated orchestration approaches offered by Kubernetes simplifies multicloud management, enabling more services to be delivered with less work and fewer errors.
Use separate clusters for different environments. Separation of environments is an important consideration for any deployment target. Ideally, you should have separate clusters for development, pre-production (staging and QA) and production.

Continuous Monitoring Practices for Kubernetes

With Kubernetes, anyone can readily view and understand the state of all resources. Kubernetes breaks down traditional, siloed visibility barriers between operations, QA, security and development teams. This improves transparency and open communication between team members.
Cloud-native applications are constructed with health reporting metrics to enable the platform to manage life cycle events if an instance becomes unhealthy.
Liveness and readiness probe capabilities of Kubernetes are used to determine the state of a containerized application.
Security monitoring and auditing capture application logs, host-level logs, Kubernetes API audit logs and cloud provider logs. For security audit purposes, consider streaming your logs to an external location with append-only access from within your cluster.

Continuous Security Practices for Kubernetes

Kubernetes supports integrated DevSecOps shift left security practices with capabilities to improve access control to container clusters, manage updates to container code and environment variables, limit resource usage on pods, control the privileges that container-based workloads run with and restrict network access across containers. This reduces the fear of security risks, giving developers, operators and security teams more confidence that security risks are reduced.
Kubernetes provides insight into what is happening within an application, making it easier to identify and fix security problems.
Kubernetes secrets objects are used to securely store sensitive data.
Enhanced orchestration controls provided by Kubernetes on deployment and on deployed containerized applications benefit from immutable consistency and improved response times.
Container images are secured to run on Kubernetes. Use security code scanning tools to check the containerized code for vulnerabilities that can exist within the container code itself, as well as in any upstream dependencies on which the image is based.
Kubernetes role-based access control (RBAC) policies are used to help guard against unauthorized access to cluster resources.
Resource quotas help mitigate disruptions caused by denial-of-service attacks by depriving the rest of the cluster of sufficient resources to run.
Pod-to-pod traffic is restricted using Kubernetes core data types for specifying network access controls between pods.
Kubernetes clusters are segmented by integrity level; for example, your dev and test environments might be hosted in a different cluster than your production environment.

Continuous Delivery Practices for Kubernetes

Enhanced orchestration controls provided by Kubernetes on deployment and deployed containerized applications benefit from immutable consistency and improved response times.
Declarative syntax used to define the deployment state of Kubernetes-deployed container clusters greatly simplifies the management of the delivery and deployments.
Kubernetes capabilities allow one container to support many configuration environment contexts. The ConfigMaps object, for example, supports configuration data that is used at runtime. This avoids the need for specialized containers for different environment configurations.
Kubernetes eliminates many of the manual provisioning and other time-consuming tasks of enterprise IT operations. In addition, the unified and automated orchestration approaches Kubernetes offers simplify multi-cloud management, enabling more services to be delivered with less work and fewer errors.

What This Means

Kubernetes is a powerful tool that enables many benefits for organizations that use it. Achieving performance, efficiently, with Kubernetes depends on following best practices. The nine pillars of practices in this blog can help organizations achieve the performance potential that Kubernetes has to offer.