Best of 2022: 4 Best Practices for Kubernetes Reliability

As we close out 2022, we at Container Journal wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

One of the main constructs linked to successful Kubernetes deployments is reliability. In the world of Kubernetes, reliability means building a stable, dependable platform so developers and DevOps teams can streamline their process and ship applications faster, with fewer headaches. To achieve this state of deployment nirvana, platform engineering teams must also partner with development to ensure workloads are configured correctly from the start, a practice many organizations fail to do well. 

Why is configuration so essential? Because Kubernetes misconfigurations are more common than you think—and can dramatically impact the security, reliability and efficiency of containerized workloads. In fact, proper cloud-native configuration lingers among the top concerns for practitioners as well as one of the most critical steps in a set of Kubernetes best practices. All in all, ensuring the health of your Kubernetes clusters via best practices means acknowledging the need for proper Kubernetes configuration. 

Kubernetes Best Practices for Reliability

In a Kubernetes environment, where distributed systems are built with microservices and containers, reliability is synonymous with stability, streamlined development and a better user experience. Along with proper configuration, these best practices should always be top-of-mind:

  1. Understand and embrace the ephemeral nature of Kubernetes. Use cloud-native architecture to help your organization grapple with the short-lived nature of containers and Kubernetes pods. Example: Instead of attempting to modify an existing container, abstract your application configuration from its container image and build and deploy a new container image through your CI pipeline. Containers are ephemeral, and running configuration management software inside of application containers adds unnecessary complexity and overhead. 
  2. Avoid single points of failure. Kubernetes improves reliability by providing repetitive components and ensuring application containers can be scheduled across multiple nodes and available zones in the cloud. Use anti-affinity or node selection to help spread your applications across the Kubernetes cluster for high availability. Node selection allows you to define which nodes in your cluster are eligible to run your application based on labels. The labels typically represent node characteristics like bandwidth or special resources like GPUs.
  3. Set resource requests and limits for CPU and memory. This step sits at the heart of the Kubernetes scheduler. If a single pod is allowed to consume all of the node CPU and memory, then resources will be starved from other pods and potentially Kubernetes components. Setting requests guarantees your application will always have enough CPU and memory to run, and setting limits on a pod’s consumption will keep pods from consuming all of the available resources on a node—an issue fondly known as the “noisy neighbor problem.”
  4. Use liveness and readiness probes. Kubernetes, by default, will send traffic to application containers right away. You can increase the robustness of your application by setting health checks that tell Kubernetes when your application pods are ready to receive traffic—or if they have become unresponsive.

Successful Kubernetes Configuration

The answer to minimizing misconfigurations is complex. While small to mid-sized businesses may find proper configuration to be more manageable, bigger companies will likely discover it’s almost impossible to manually check each security configuration to assess its risk. Because Kubernetes defaults tend to be inherently open and not secure, it is important to avoid using these default settings until all the security implications—and their impact on overall risk tolerance—are clearly understood.

Helpful guidance and a useful framework for hardening an environment can be found in various objective, consensus-driven security guidelines for Kubernetes software, such as the CIS Benchmark and NSA Hardening Guide. When these best practices are paired with risk-based policies integrated into the CI/CD pipeline, container security improves. Commits or builds that do not meet minimum security requirements or provide guardrails for developers can be halted.

Protecting Kubernetes clusters and workloads at runtime to ensure security, efficiency and reliability demands a multi-pronged approach using defense-in-depth. Part of this solution comes from finding a SaaS platform with the ability to establish effective governance, streamline development and operations and provide a better (and safer) user experience.

Because misconfigurations are so common, building a stable, reliable and secure cluster only happens when the best practices outlined here are followed. And this level of governance only comes through a trusted partner, well-versed in the process of unifying teams, simplifying complexity and building on Kubernetes expertise to save time, reduce risk and configure with confidence.

Robert Brennan

Robert Brennan is director of open source software at Fairwinds, a cloud-native infrastructure solution provider. He focuses on the development of open source tools that abstract the complexity from underlying infrastructure to enable an optimal experience for developers. Before Fairwinds, he worked as a software engineer at Google in AI and natural language processing. He is the co-founder of, an open source platform for building API’s and integrations, and LucyBot, developer of a suite of automated API documentation solutions deployed by Fortune 500 companies. He is a graduate of Columbia College and Columbia Engineering where he focused on machine learning.

Robert Brennan has 14 posts and counting. See all posts by Robert Brennan