Solving Container Security Challenges: Vulnerabilities, Shadow Deployments and Deployment Gaps

Kubernetes is now more than a decade old. It was started as a management tool for container deployments and is now responsible for orchestrating cloud-native applications at scale around the world. As a Cloud-Native Computing Foundation (CNCF) project, Kubernetes has more than 3,500 contributors. According to the CNCF Velocity report, it is the second-largest open-source project after Linux. Kubernetes and cloud-native applications go hand in hand. According to the Pure Storage Voice of Kubernetes Expert Research report, 80% of organizations think that most of their new applications will be built on cloud-native platforms over the next five years. 

While these applications might be easier to deploy, they also must be secured — these environments must be protected. This area is not as well developed as might be expected. So, why do these risks exist, and what can we do to solve these problems? 

Security Risks in Cloud-Native Environments 

In containerized environments, there are three main areas where security risks can exist — vulnerabilities in software, shadow containers and deployment problems. Each of these can contain flaws that attackers can exploit to get into the infrastructure of an organization. 

Software vulnerabilities are one of the oldest and most common potential risks for IT environments. According to the Verizon Data Breach Investigations report, 14% of security breaches were linked to vulnerabilities as the initial point of access. While dealing with vulnerabilities is necessary, fixing the problems is not as easy. New issues are discovered all the time, while existing container images may hold vulnerabilities that are yet to be addressed.  

To protect against these software issues, you must know what issues exist within your containers and how many of those images contain problems. Once the list of affected container images is compiled, you can decide which images and deployments are risky and need to be updated. Similarly, the images in the container registry can also be fixed, so future deployments are secured by default. 

This is a continuation of a traditional security process, updated for a modern cloud-native deployment approach. You must track your implementations in practice and apply best practices for each cloud. This set of insecurities is not based on any flaw in the software; instead, it is based on errors in how the software and infrastructure components are configured. This set of flaws is responsible for a similar level of security risk to software vulnerabilities. According to the Verizon DBIR report, around 13% of issues can be tracked back to misconfigurations. 

Each major cloud environment has its own benchmark for secure deployments. While there are some similarities around account security and reducing privileges, each cloud will have its own set of rules to follow to deploy, manage and harden cloud workloads. The Center for Internet Security Benchmarks provides in-depth guidance for major hyperscaler cloud providers, but it is all too easy for developers to overlook these practices as they deploy container workloads. As these images go through the software development pipeline, they can hold risks that can be exploited in the future. 

The biggest risk of all around containers is due to workloads that are not on the asset list. When developers implement workloads in containers, they should include any default security implementations within those images. However, if that does not take place, then the containerized application will run without full oversight or security teams checking potential problems. Over time, this ‘shadow container’ deployment can have more outdated software or misconfigurations within the images involved. This leads to more risk and more work in the long term for the developer. 

So, why do these issues exist in the first place? For developers, their goals are to deploy new updates faster. Anything that gets in the way will affect their ability to meet key performance indicators (KPIs) on developing and deploying those new software packages into containers. Yet this emphasis on speed can affect the overall pipeline and lead to security issues getting in. 

For security, the goal is to reduce risk. Yet this action can affect how developers work and lessen their pace of deployment. In an argument between security and developers, developers tend to win as their work is directly linked to revenue. So, security teams have to work with developers and embed their approaches into the overall software development life cycle (SDLC). In doing this, security can keep developers productive while keeping that eye on risk. For developers, using security teams to reduce potential risks and keep guardrails in place can make them more efficient by reducing application rework. 

Detecting Container Security Issues Requires Multiple Techniques 

To solve these container security issues, you must adopt multiple approaches. While software vulnerability management is a long-standing discipline that the majority of security professionals have spent years reviewing, the same approach cannot be applied to containers.  Detecting issues within container images can rely on both agents being installed and agentless scanning. 

Agents can tell you what is installed within each container, while agentless approaches allow you to cover the whole application, typically by making a copy of the containers which can be scanned without affecting performance for the production deployment. Ideally, using both agent-based and agentless approaches should give you a fuller picture of what is deployed and any potential risks. 

While these methods can help you spot potential issues inside your known application environments, what about those shadow containers that have not followed best practices during deployment? These instances will leave traces in your cloud environment, which can then be detected. Finding these containers is the first step, followed by ensuring that they have necessary security tools installed. 

For security, this overview of assets and risks provides a full picture of what is in place and where changes are made. This list can then be passed over to the development team or IT operations unit, which will apply the changes in practice. However, this can have its own problems. 

While security tools can take you so far, there is another issue to consider — who owns those applications over time? While an IT asset inventory can tell you what exists and what is installed, it does not generate ‘why’ something was created or who is responsible for it over time. This trail back to those who developed an application should not stop there — developers only build for the business because there is a potential need. Therefore, going back to the line of business team as part of the process should also be on your list. 

Getting the right information over is also essential. Security might want to provide a list of all the issues that exist across applications, but this list can run to thousands of potential vulnerabilities or misconfiguration changes. On its own, this list does not help developers know where to focus their efforts. Instead, using risk data to highlight what priorities are most important and pressing can help developers know where to focus.  

This idea of responsibility around containers is essential, because they can exist for very short periods based on demand. Getting ahead of security risks like out-of-date images or shadow container deployments is a necessary task. But it also flags the need for better collaboration across teams. Security and software development should not be at loggerheads around these issues. Security should flag issues while also providing risk and context data about the most pressing problems. For developers, adding security into the CI/CD pipeline and ensuring that images follow secure deployment guidelines will reduce rework. 

As the majority of new applications are deployed in cloud-native environments, we must improve security around those cloud and container deployments. That does not mean just adding more security tools, as necessary as those are, instead, it means we must develop the risk operations framework that can assess potential problems and provide context across teams to fix the vulnerabilities or misconfigurations easily. Centralizing risk operations and helping all teams to understand, process and manage risk around cloud-native deployments will be essential. 

KubeCon + CloudNativeCon EU 2025 is taking place in London from April 1-4. Register now.