A hacker exploiting your baseboard management controller (BMC) or other firmware running largely unnoticed on a motherboard is every kind of bad news – and yes, it happens in the wild, says Yuriy Bulygin, CEO of Eclypsium; listing off examples with the comfortable experience of someone who has been tracking this distinctive landscape for a long time.
Bulygin founded Eclypsium in 2017 after a decade at Intel, where he ran its security threat analysis and research team. A naturalised American citizen, he grew up near Chernobyl, Ukraine, before emigrating to the Portland, Oregon area. The softly spoken CEO, who unwinds by training in Brazilian Jiu-Jitsu, launched the company to focus on what he refers to as “infrastructure supply chain security”; a category that involves thinking about cybersecurity from the hardware and associated firmware level up.
As he recalls from his days at Intel doing vulnerability analysis on wireless communications and drivers, there was an “understanding at the time that ‘hardware was hardware’ – but today software has eaten the world and it has eaten hardware; every component, every chip, every device runs a lot of software, millions of lines of code, and it all has vulnerabilities.”
That lesson is increasingly being learned the hard-way in some quarters, but recognition is dawning that firmware and the infrastructure supply chains deserve far more security attention – recognition that has seen Eclypsium build increasingly deep relationships with customers including federal agencies, hardware OEMs and finance companies, among others.
Trust, but verify?
Eclypsium has now analysed over 11 million types of firmware – software embedded in a piece of hardware that makes it run; building out its coverage at the rate of nearly a million more forms of firmware yearly.
“You just don't necessarily want to trust manufacturers,” Bulygin tells The Stack in the company’s Portland, Oregon offices – speaking amidst the hum of testbed network appliances, and surrounded by stacks of motherboards and other hardware. “A manufacturer uses hundreds of suppliers, and one of those suppliers might have been compromised, and malicious or tampered components end up in any device,” he says.
An example? He points to 2019’s “Shadow Hammer” attacks on the ASUS Live Update utility (pre-installed on new ASUS computers, for automatic BIOS, UEFI and driver updates) – and emphasises the poor code running on so many “black box” network appliances as well, which has caused a flurry of recent critical security issues and widespread exploitation.
Eclypsium Automata
In terms of product, Eclypsium has built a software engine that can identify and spot both vulnerabilities and malicious behaviour across not just laptop and/or server firmware, but cloud environments, VMs, and networking devices running at a level that no EDR will ever notice (not least because EDR doesn’t run on “black box” VPN appliances.)
As Eclypsium’s CEO puts it: “We built a number of binary analysis engines that we've recently combined into one engine that we call Eclypsium Automata in a pipeline-like approach where we do static analysis and we do dynamic analysis of every binary that we extract from every device.
“We run this against models that we trained on firmware data; we also run multiple functional similarity engines, where we can take a binary and see ‘Okay, so that binary is unknown, but it looks like it's an NTFS driver, or it looks like it has USB functionality…’ so we decompile binaries, we disassemble binaries and run a lot of dynamic analysis on those,” he tells The Stack.
“But the bottom line is that when we analyse a binary, we can determine (even if it was a ‘known good’ from a manufacturer) if it's really good or is exhibiting strange behaviour or has some kind of malicious functionality.”
A once arcane security category goes mainstream
Eclypsium’s early customer base included highly security-aware organisations like federal agencies and hedge funds, among others. But the infrastructure supply chain security category it pioneered and customer interest more broadly are going distinctly mainstream.
Thank, in part, federal agencies – with CISA and the NSA in June 2023 for example, urging enterprises to pay BMC security far more attention and also emphasising that the risk of persistent rootkits (of the kind that bypass UEFI Secure Boot) like Black Lotus was going overlooked.
The two wrote: “BMC firmware is highly privileged, executes outside the scope of operating system (OS) controls, and has access to all resources of the server-class platform on which it resides. It executes the moment power is applied to the server. Therefore, boot to a hypervisor or OS is not necessary as the BMC functions even if the server is shutdown.”
“A vulnerable BMC [provides] malicious actors the opportunity to employ tactics such as establishing a beachhead with pre-boot execution potential…a malicious actor could disable security solutions such as the trusted platform module (TPM) or UEFI secure boot, manipulate data on any attached storage media, or propagate implants or disruptive instructions across a network infrastructure,” the agencies warned CISOs.
They added that most organisations do not even take “the minimum action to secure and maintain BMCs. Hardened credentials, firmware updates, and network segmentation options are frequently overlooked…”
Pre-auth RCE in a BMC...
Troublingly, critical security vulnerabilities here are rife and often go widely overlooked. Last year, for example, Eclypsium Research discovered and reported five vulnerabilities in American Megatrends (AMI) MegaRAC BMC software that is found in tens of millions of devices worldwide and used by numerous OEMs to deliver “lights-out” management for servers.
The bugs, identified in code that leaked after an AMI ransomware incident, gave an attacker unauthenticated remote code execution (pre-auth RCE) and device access with superuser permissions.
Over the years Eclypsium has also attracted industry attention with some compellingly distinctive security research designed to highlight industry vulnerabilities. Two examples stand out to The Stack, one older, one more recent. The older one involved the company compromising the BMC of a cloud server it had leased and retaining access to it even after the compute was re-released back into the provider’s pool of capacity; allowing it sustained access to the cloud provider’s customers.
The other, more recently, saw Eclypsium researchers tear down an Ivanti Pulse VPN appliance in the wake of a series of breaches, to find it was running:
- Linux kernel 2.6.32 (end of life in February 2016)
- OpenSSL 1.0.2n (December 2017)
- Python 2.6.6 (August 2010)
- Perl v5.6.1 built for i386-linux (not x64, April 2001)
- Bash 4.1.2 which, surprisingly, has been patched for Shellshock
- A number of outdated libraries with known CVEs and exploits
As Yuriy, running a finger over every component of a motherboard to explain what firmware runs where and what the risks typically are, concludes simply: “My main goal is for my customers to understand everything about every critical device…what code is it running internally, how was that code built, who built it and what practices did they use? I want my customers to have that understanding for every piece of equipment that they rely on in their infrastructure.” Learn more here.
Delivered in partnership with Eclypsium.
