The State of K8s Software Supply Chain Attacks
Securing the software supply chain is in the zeitgeist, and for a good reason. Software supply chain attacks grew by more than 300% from 2020 to 2021. Supply chain attacks continue to surge in recent months, targeting open source software, container images and packages within the CI/CD pipeline. As a result, malicious container images continue to rise and security researchers have noticed a change in focus from Docker to Kubernetes-based attacks.
Tracking Software Supply Chain and Kubernetes Attacks, a study conducted by Aqua Security’s Team Nautilus, explores the state of cloud-native cyberthreats. The team used a combination of container image scanning and honeypots to lure attackers. They discovered increased attack sophistication across the board, increased focus on CI/CD environments and vulnerable Kubernetes deployments and increased malware and cryptomining packages, which could be used for cryptojacking.
Kubernetes, now used by 5.6 million developers, is a popular CNCF open-source package for container orchestration. With so much reliance on this platform, it’s increasingly essential to harden its use and avoid misconfigurations. Below, I’ll explore critical takeaways from the Aqua Security report to learn what cloud-native operators, especially Kubernetes platform owners, should watch out for.
Container Images Attack Types
Out of 1,1000 container images examined, 13% were related to potentially unwanted applications. Cryptominers were the most common type of malware observed in the wild. More than half (54%) of compromised container images also were found to have backdoors, enabling hackers to access a system remotely and establish a persistent connection.
The report found that 50% of container image incidents had worms in 2021; these worms enable hackers to automatically seek vulnerable systems and infect them, significantly increasing the potential attack surface. Attackers have also been found to run rootkits, which can be leveraged to hide other nefarious behaviors.
Even the slightest misconfiguration can compromise a system. For example, using the right scanning tools, threat actors can detect vulnerable Docker APIs in just under one hour. Although threats to Docker APIs continue to be a risk, coordinated hacking groups have shifted much of their efforts from misconfigured Docker APIs to Kubernetes, the report found.
Kubernetes Becomes the New Target
A full 19% of malicious container images analyzed targeted Kubernetes—specifically kubelets and API servers. This is bad news, as Kubernetes administrative control can grant much lateral movement. This weakness arguably has broader implications as opposed to a single container exploit. This threat is underscored by the fact that there are now 380,000 publicly deployed Kubernetes API servers, according to the Shadowserver Foundation. While this doesn’t necessarily mean all of those API servers are vulnerable, it does highlight how vulnerable these public instances are—if misconfigured, they become low-hanging fruit for an attack.
“While a defense-in-depth strategy is always beneficial, it must also be balanced against the threat models created for the software and its intended usage profile,” says Tim Mackey, principal security strategist, Synopsys CyRC. “API endpoints, like those of the Kubernetes API, are intended for public consumption, though certain usage patterns might dictate blocking API ranges, implementing rate limiting or even the use of a VPN.”
Another emerging cloud-native threat vector concerns user interface (UI) tools used to access Docker and Kubernetes. These operational views provide things like visibility into a cluster or a visual control point for managing access. Now, hackers are adapting their techniques to discover holes in these systems—specifically, the dashboards that don’t require authentication. This poses a significant threat, as bad actors could steal keys, change settings or start running containers and deployments.
Cloud-Native Supply Chain Threats
Nowadays, many CI/CD tools are used to automate app development. Although open source projects greatly aid the development process, they pose an increasing threat to the software supply chain. Hackers can explore these open source packages for vulnerabilities or even poison them with nefarious commits to compromise the integrity of the CI/CD pipeline or manipulate a build.
Security researchers have proven this is possible. For example, security researcher Alex Birsan recently documented a “dependency confusion attack.” He placed malicious code with the same name as popular packages in an official-looking repository. “Squatting valid internal package names was a nearly sure-fire method to get into the networks of some of the biggest tech companies out there; gaining remote code execution and possibly allowing attackers to add backdoors during builds,” he writes.
According to the report, out of 30,0000 Python packages scanned, 170 had suspicious activity. To further substantiate the supply chain issue, Aqua Security created a honeypot with an unpatched Log4j CVE, which was hit with at least a dozen attack types within minutes. Of these, 67% were reverse shell attacks. The experiment proves how large botnets have evolved to understand and rapidly leverage exploits in popular open source packages.
Final Thoughts
Today’s new exploits are tomorrow’s common threats. Thus, cloud-native operators need to stay on top of potential vulnerabilities and work to limit misconfigurations across platforms like Docker and Kubernetes. Part of this will be assessing the provenance of packages to ensure they don’t pose a threat — something Sigstore is aiming to achieve. Some further tips to mitigate threats include:
- Correct misconfigurations of cloud services
- Ensure K8s UI tools aren’t publicly accessible
- Harden multi-cluster Kubernetes environments
- Ensure runtime security
- Adopt a layered K8s security strategy
- Implement scanning in development
The Tracking Software Supply Chain and Kubernetes Attacks is a thorough report that investigated images from public registries and even conducted honeypot testing to lure attackers into attacking in the wild. For more in-depth information, you can pick up a copy here.
