DevSecOps Essentials for Containers

DevSecOps has emerged as the preferred and necessary approach to secure modern applications by ensuring security practices are seamlessly integrated into the software development and delivery process in an automated fashion. While companies have historically invested in security, many have not progressed far in their efforts to program security into their software development and operations life cycle. 

While container-based applications bring many efficiencies like speed, flexibility and agility, these are often not realized if manual security configurations, reviews and audits stand between development and production. As such, taking full advantage of modern approaches to application development also requires a modern security approach. 

An ongoing challenge in securing application services is preventing vulnerabilities from knowingly or unknowingly being introduced, both during development and eventual deployment, and to validate that every workload deployed to production stays compliant. This is where DevSecOps comes in.

For those just getting started with adopting DevSecOps, here are essential best practices to implement as code in the various stages of the modern application life cycle, helping you adopt a practical DevSecOps approach for building secure modern apps. 

Best Practice #1: Make developing secure, cloud-native software easier by using API microgateways to complement programming language frameworks

Programming language frameworks make adopting recommended security practices and patterns easier for developers. Frameworks will help development teams create secure applications by default, and many already include numerous security features like authentication integration, circuit breaker patterns and protection against attacks like cross-site request forgery. These frameworks also often include configuration management within the framework itself. Additionally, many can connect to an external configuration service for the centralized management of policies. 

Building atop language frameworks, the emergence of API microgateways adjacent to every microservice can help enforce security access controls and governance requirements in a consistent way for polyglot applications; i.e., the microservices are written in different languages. 

Best Practice #2: Build modular container images with layers that can be individually updated, scanned and verified 

While traditional applications have updates applied directly while the app is running, the process for containerized apps is entirely different—and, in fact, results in an entirely new container image that needs to be verified, tested and deployed. Technologies like Cloud Native Buildpacks (CNB) make it possible to structure the contents of container images and quickly patch entire fleets of those images automatically. The CNB project provides associated tooling to transform source code into container images using modular, reusable build functions. It’s important to note that to accomplish this, CNB takes advantage of advanced features in the OCI image standard that are underutilized by the Dockerfile model; as such, the buildpacks model is considered superior to the Dockerfile approach.

The Paketo ( project incorporates these practices to enable organizations to build and reason about robust deployments and updating practices for containerized environments, one that allows for continuous updating and offers metadata proving the trustworthiness of the software within.

Best Practice #3: Adopt control policies for images in your environment and centralize control of container updates and patches

A private container image registry allows teams to store container images or other binaries securely and privately. Within it, operations teams can validate images, set usage policies and keep images refreshed and updated so that developers can leverage them. For third-party or open source applications, companies often pull images from public repositories for those applications and, in turn, run them on critical production infrastructure. Whether from unpatched CVEs or from malware embedded within the image layers, these organizations are ultimately exposed to any number of threats.

Best Practice #4: Continuously update the infrastructure layer and centralize control of patches

The frequency at which new infrastructure is deployed may be close to how often applications are redeployed. This means Kubernetes clusters may be constantly provisioned, deleted and recreated. Securing numbers of Kubernetes clusters and ensuring they include ubiquitous and uniform policies is a major challenge. At the same time, software releases for major open source Kubernetes upgrades are released about every six months, with a consistent stream of patches coming in between. These upgrades can become burdensome for platform teams, which prevents them from deploying the most recent updates and patches consistently.

A centralized update management system can enable an organization to maintain security policies set by their security team—and make the process of creating new and fully up-to-date Kubernetes clusters easy and automated. From a single pane of glass, it becomes easier to identify when clusters across the organization require upgrades. This central control also confers the ability to elastically repave infrastructure to ensure any given implementation does not become stale.

The Cluster API project is a great example; focused on providing declarative APIs and tooling to simplify provisioning, upgrading and operating multiple Kubernetes clusters.

Best Practice #5: Secure inter-container communications

Modern applications are broken into services, each of which must, in turn, communicate with one another to facilitate the look and feel of a full application to the end user. However, these communications typically occur over untrusted mediums which can be vulnerable to outside threats such as “Man-in-the-middle” (MITM) attacks and should be commensurately secured and monitored.

The process of securing inter-service communication starts by identifying every service that is to participate in the communication. In order to accurately determine these identities, the provenance of the binary that the service is being deployed from must be inspected. There are a number of questions the system must be able to answer to determine that integrity and confidentiality haven’t been tampered with and the service is authenticated. Is this service instantiated from a binary in a trusted repository? Is such binary signed? What event triggered the creation of this service? Did Kubernetes, in fact, schedule this service?

A recommended practice is to deliver workload-specific, short-lived, automatically rotated keys and certificates as opposed to shared reusable API keys or long-lived tokens. 

Once services possess credentials conferred as the result of attestation, these can, in turn, be leveraged to encrypt data-in-motion. Projects like SPIRE can attest running software systems and issue cryptographically proven credentials. This, in turn, allows two workloads or services to establish trust between each other; for example, by establishing an mTLS connection or by signing and verifying a JWT token. 

Using these essential best practices in the various stages of the modern application life cycle can help you adopt a practical DevSecOps approach for building secure modern containerized applications.

Ajay Patel

Ajay Patel is the General Manager for VMware’s Modern Applications & Management Business, with the mission to be the leader in application modernization, cloud-native application development and multi-cloud management through VMware Tanzu and vRealize portfolio. He has over 30 years of enterprise software expertise and holds a Bachelor’s of Science in Electrical Engineering and a Bachelor’s of Science in Computer Science from the University of California, Irvine. Ajay previously served as the Treasurer and board member for Cloud Foundry Foundation. He is also a board member and technical adviser to several private and venture-funded software companies.

Ajay Patel has 1 posts and counting. See all posts by Ajay Patel