Containers, often managed using Kubernetes, have given companies a highly scalable and agile way to accelerate code from development to production, as well to decouple applications from infrastructure.
The rapidly rising adoption of containerised microservices and on‑demand cloud services is demonstrated in the median number of containers per host, which has grown nearly six-fold in the last six years.
For all the significant benefits of this cloud-native approach, it also creates new attack surfaces for malicious actors to exploit, and many development and SecOps functions are still getting up to speed on the threat.
The Sysdig's 2023 Cloud‑Native Security and Usage Report highlights how extensive this threat surface really is: 87% of container images running in production have a critical or high severity vulnerability, and 71% of vulnerabilities have a fix available that hasn’t been applied. These vulnerabilities are being introduced in ever-greater numbers through software supply chains, the cloud and container security specialist found.
Register now for a free online masterclass on container security with Sysdig and The Stack, April 27, 16:00 GMT+1
That risk comes into clear focus given that over 50% of images are still pulled from public repositories; a number that has reduced notably over the past year as security maturity improves but which remains high.
The vast scale of the SolarWinds breach in December 2020 demonstrated to the world just how sophisticated and wide-reaching the threat of supply chain attacks had become, and attacks targeting the software supply chain are still on the rise today. Recent examples include the Iran-backed hack of a US federal agency that failed to patch against Log4Shell. (The attackers exploited an “unpatched VMware Horizon server, installed XMRig crypto mining software, moved laterally to the domain controller, compromised credentials, and then implanted Ngrok reverse proxies on several hosts to maintain persistence” according to CISA and the FBI.)
“There’s a constant cycle of news about ransomware attacks and data leaks, but you don't hear much about the massive exploitation of containers,” says Michael Clark, Director of Threat Research at Sysdig, which uses cloud-native runtime insights (including data pulled using either kernel modules or eBPF probes) to help companies detect, prevent and respond to cyber threats. “Everybody’s using containers and Kubernetes – and breaches are happening, but organisations just aren’t meaningfully aware of it as a considerable threat vector.”
Container security: Threats lurking in the software supply chain
A key reason why these attacks are avoiding detection is because containers are generally less vulnerable to the attack path that IT teams are accustomed to, whereby malicious code is inserted remotely via the internet. While this method could theoretically work in some instances, most containers are not exposed directly to the internet.
It is in the software supply chain, rather, whether attackers find a better route to exploiting containers, through both public images and private images. All of the software and dependencies that developers need to create containerised applications are in these images, so if an attacker can get malicious code into one that is then used in a container deployment of, for instance, 100,000 nodes, the attacker has access to all of those nodes.
See also: Securing cloud-native architecture at scale: From runtime to remediation
“This is where the real risk for containers lies,” says Clark.
“Vulnerabilities in public images, in particular, are rife. As a threat research function, we run these public images through a specialised sandbox which looks for malicious behaviours, instead of just vulnerabilities.”
He adds: “We've identified tens of thousands of images that have either malicious behaviours or dangerous behaviours that you wouldn't want to have a developer load on your network. In our threat report last year we found definite evidence of malicious packages disguised as legitimate packages in public images. They may have a character off, or just the extended version of PyTorch, disguising themselves in the hope that a developer will accidentally load them. Once they do then they’re basically inside your network. Often it is cryptominers but we’ve also seen remote access Trojans and all sorts of other nastiness.”
Focusing on the real risk with runtime visibility
The complexity of these vulnerabilities and growing reliance on open source software and third-party code is exposing a lack of maturity in DevSecOps processes, which continue to lag behind the overall pace of cloud adoption.
The rise of DevSecOps roles shows companies now understand that by shifting security left they can assess code at the earliest stage of the pipeline. However it’s not practical to fix every single vulnerability discovered in images when multiple workloads are being maintained at scale.
Ultimately, says Sysdig’s Michael Clark, “the source of truth is what it does when you run it. That's why runtime visibility is so vital. You need observability to see what your packages are doing or if they change behaviour. Seeing these behaviours at runtime is the best way to know your dependencies are safe.”
Sysdig says the capabilities it offers allow users to identify not just unsafe images but “bloat” in images that drives up cloud costs, as well as unused Kubernetes resources. Supply chain security that also generates ROI in reduced cloud infrastructure costs? In a world of increasing FinOps focus, that sounds like a real win-win.
