Closing the Gap With ITDR for Cloud-Native Security and Kubernetes RBAC

March 21, 2024March 20, 2024 Jimmy Mesta cloud native security, identity, ITDR, kubernetes, RBAC, role-based access control

by Jimmy Mesta

Compromised identities have played a critical role in multiple security breaches that are in the headlines, demonstrating just how indispensible identity threat detection and response (ITDR) has become to security teams. But, in many organizations, ITDR has a major gap when it comes to cloud-native and Kubernetes role-based access control (RBAC.)

Overly permissive RBAC identities were at the center of three of the four major attacks targeting Kubernetes in 2023. This year, software supply chain attacks targeted kubeconfig files for the first time – with a recent survey showing that 58% of teams using Kubernetes had a security issue in the last 12 months with insufficient access controls in their Kubernetes environment.

So what is required to apply ITDR to cloud-native security with Kubernetes RBAC?

What is Identity Threat Detection and Response (ITDR)?

ITDR is one of the top security and risk management trends, according to Gartner. ITDR tools detect potential security incidents involving user identities and offer a range of responses to mitigate risks. When a malicious identity is flagged, responses can include right-sizing, deletion, blocking or other appropriate actions.

In a zero-trust program, ITDR is the ultimate way to check for anything that might have gotten through the guardrails and least privilege access policies your organization has in place. ITDR tools also provide strong recommendations, based on malicious activity, for improving identity posture. All signs point to the prioritization of ITDR as a part of zero-trust initiatives this year. Recent research puts identity and access management (IAM) and cloud infrastructure as the top two areas of focus for CISOs in 2024.

Unique Challenges of ITDR with Kubernetes RBAC

The advent of cloud and cloud-native technologies, including containers and Kubernetes, has exponentially increased the number of identities, expanding the scope of security teams’ detection and response efforts. In particular, Kubernetes RBAC presents a unique challenge due to its complexity and the absence of clear ownership between developers and security teams.

Challenges of ITDR with RBAC include:

Ambiguous ownership: To properly secure RBAC, somebody needs to write the policies, audit them, do user access reviews and investigate in the case of an incident. Most teams agree that the security teams should be performing the audit and investigation, while user access reviews are generally a shared task as security checks its data with engineering. However, who is responsible for writing the policies will differ across organizations.
Default toward over-permissions: When trying to ship applications fast, it’s more practical to purposefully over-permission rather than under-permission so nobody is blocked.
Complexity and Kubernetes Security skills gap: The RBAC policies themselves are hard to write, requiring a level of nuanced YAML, and there is an inherent risk involved when they are written by platform teams or somebody without security or least privileges in mind.
Increased Number of Digital Identities: Keeping track of and securing the volume of digital identities today stretches security teams to their limits.
Software Supply Chain security in the Kubernetes ecosystem: Many Kubernetes tools require high privileges within RBAC in order to function, as they are deployed in-cluster as pods, daemonsets and plugins. This expands the attack surface through the hundreds of software supply chain tools in the Kubernetes ecosystem.
Misconfiguration Risks: Misconfigurations in components like kubelet or out-of-date versions of clusters can open the risk to vulnerabilities and attacks.
Insecure defaults: There is no built-in RBAC query mechanism in Kubernetes to check for over-permissions or help you write policies, and there are many default permissions that can be introduced just be deploying into managed cloud Kubernetes services like AKS, GKE or EKS.

Importance of ITDR in Mitigating Kubernetes RBAC’s Role in Cloud-Native Attacks

In the case of a cloud native attack, ITDR is the only solution that could discover or mitigate the large role played by RBAC in cloud native attacks.

In 2023, a new software supply chain attack targeted Kubeconfig files, containing authentication data for Kubernetes clusters, allowing access to user, namespace, and API info. This exposed system:masters group role bindings, providing numerous ways to breach clusters.

Many Kubernetes management tools demand high permissions, often deployed via Daemonsets across multiple nodes, spreading permissions widely. A recent report identified tools with excessive permissions and efforts to mitigate risks. For instance, EKS and AKS used a Daemonset to update nodes, necessitating the ‘update nodes’ permission. However, without scoping permissions down to the pod level, Kubernetes would allow every node in the Daemonset to execute ‘update nodes.’ This vulnerability could enable a malicious pod to incapacitate others while retaining undue permissions and control.

In the 2023 Dero and Monero cryptocurrency miner attacks on Kubernetes, attackers leveraged RBAC permissions to execute a Daemonset and create malicious pods. They exploited Kubernetes APIs with –anonymous-auth=true to gain entry, requiring RBAC configurations allowing pod creation.

A cross-tenant attack on ACI, a Container as a Service (CaaS) platform, was identified, enabling attackers to infiltrate other users’ containers. This attack relied on accessing privileged Kubernetes Service account tokens to control the API server and the entire cluster.

FluentBit privilege escalation poses a concern for GKE, as it’s the default logging tool installed on each cluster node. Vulnerabilities in FluentBit containers could be exploited to read service account tokens and access pods’ tokens on the node.

Before vendors and researchers had found and publicized their fixes and mitigations for these issues, the only thing that could have helped identify an attacker taking advantage of these issues would have been an ITDR solution for Kubernetes RBAC. And in this context, an ITDR solution would need to understand the usage of valid permissions, prioritize risks based on a holistic view of the threat landscape, including runtime events, and in the best case scenario, be able to right-size identities.

Posture Management With CIEM and KIEM: The Legacy Approach to Securing K8s RBAC

Historically, RBAC security tools were designed to enhance identity posture and implement hardening strategies. While Cloud Identity and Entitlements Management (CIEM) focuses on Cloud IAM, Kubernetes Identity and Entitlements Management (KIEM) is solely focused on posture for Kubernetes RBAC, with the following capabilities:

Over-Permissions: Managing RBAC involves adhering to security best practices, such as applying the principle of least privilege.
Misconfigurations related to data breaches: Risks include unauthorized access to sensitive data, such as the ability to read secrets, mount volumes, and get access to tokens with privileges.
Misconfigurations in the control plane and nodes: Misconfigurations in the control plane and node components, such as kubelet, or using outdated cluster versions can expose vulnerabilities. An example of this is an admin cluster giving anonymous users excessive permissions.
Privileged Access: Users may have access to privileged pods and powerful permissions controls allowing them to execute high-level commands,, mount resources, or impersonate other privileged users.
Advanced controls: To validate advanced Kubernetes features such as admissionController, certificateSigningRequest requires sophisticated controls.

But posture is not actual behavior, nor is it a checklist for RBAC security. With ITDR, you can expect more than just laundry lists of permissions. You can expect to understand the actual usage of valid permissions, how to right-size, which identities are unused, and which identities are top priority based on their relationship to other contextual factors like runtime events, image CVEs, and more.

ITDR Best Practices Checklist for Kubernetes RBAC

So what do you have to do if you truly want to detect and respond to any compromises of the Kubernetes RBAC identities across your environment? Ensure that any tool you use can match the following ITDR best practices checklist for Kubernetes RBAC, not just cloud IAM.

Checklist

Risk Score: There should be a way to understand the holistic risk of an identity, including context from runtime, cloud, image CVE and K8s misconfiguration.
Admission Control: One of the best tools for setting guardrails in Kubernetes RBAC is admission control, an ITDR must have access to this critical response tool.
Top Risky Identities: Use the risk score to understand potentially malicious, high priority identities.
Actual Usage of Identity: Combine over-permissions and audit logs for failed/successful access attempts.
Unused Identities and Permissions: Filter for used or unused identities, with right-sizing recommendations.
Namespace/env Filters: Filter out results from any namespace for easier investigation.
Cover Service Accounts and Users: Include K8s Groups, Service Accounts and Owners.
Keep Track of Remediation: Create progress reports.
RBAC Right-Sizing: Streamline the implementation of least privilege access with detailed right-sizing guidance that takes into account the identity’s behavior