If there is one thing that keeps the CIOs and CISOs of water utilities up at night, it is the fear of a successful attack on their IT infrastructure. On February 5 these fears briefly crystallised as a worker at a water treatment plant in Oldsmar, Florida, watched as a remote user took over his mouse, then ramped up control settings to inject 11,100 parts per million of caustic soda into the city's water supply, rather than its standard 100 parts.
The plant operator manually undid the move and the water supply was not affected. (Caustic soda, or sodium hydroxide, is used as a pH adjusting chemical in the treatment of drinking water to control the corrosion of metals such as lead from pipes into the drinking water). The plant's operators say other controls would have prevented the water actually being poisoned, but it was not immediately clear if these were also remotely accessible and hence vulnerable to being switched off maliciously.
https://www.youtube.com/watch?v=MkXDSOgLQ6M
“The protocols that we have in place, monitoring protocols, they work — that’s the good news,” said Oldsmar Mayor Eric Seidel at a press conference called on Monday. “Even had they not caught them, there’s redundancies in the system that would have caught the change in the pH level."
The plant was using TeamViewer to allow remote access, Bob Gualtieri, the sheriff of Pinellas County, told Wired. How they got access to it remains unknown.
The FBI is now investigating. City officials said the plant operator had at first not been concerned to see the sudden remote access, as their supervisor regularly remotely accessed the plant's controls. They did not mention how this remote access was made or secured. Multiple sectors -- particularly across critical national infrastructure -- will be watching for a detailed post-incident response/RCA of the intrusion in the near future.
Read this - NIS 2: Winds of change blowing as EU tightens up cybersecurity requirements for CNI.
Security researchers say that a quick look on Shodan, the search engine for connected devices, reveals plenty of exposed water treatment plants; e.g. with networked equipment plumbed in with no passwords.
Former General Motors CISO Rich Armour told The Stack's founder Ed Targett in 2020 that asset visibility alone was a major issue for industrial service providers or utilities looking to secure their networks.
As he put it: "Inside General Motors, when we ran tests against some of our complex factory environments, we found that the vast majority of industrial control security vendors were not able to identify a significant proportion of the industrial control systems we had in our factories. It’s a tough environment because of the huge variety of industrial controls that exist out there: for robots to control motors, to PLCs to lifts, automated torque wrenches; the variety is almost endless. And in that, I think poses a significant challenge."
Armour added: "The other dimension of it is that unless those controls and their network activity has been observed in a laboratory or testing environments, then it’s also difficult to develop signatures. And so there are a lot of challenges. And unfortunately, it is a critical part of the industrial control security function to certainly understand what assets are out there”.
Another issue is that sensors/networked infrastructure is often incredibly physically dispersed. As the former GM CISO put it: “The simple reality of how large these facilities are and how widely dispersed the control systems are, it makes physically inventorying them very, very difficult…You can imagine the massive sprawl of an oil refinery, for example. These control systems will be not only all over the the refinery campus, but they’ll also be, in some cases, very inaccessible locations: very high up in physical infrastructure. Sometimes they’re underground.”
Stuart Reed, UK Director, Orange Cyberdefense noted in an emailed comment: "The attack against Oldsmar's water supply is precisely the kind of assault on critical national infrastructure that cybersecurity experts have been fearing for years. Key facilities worldwide are constantly being probed for weaknesses, and there are still significant concerns about the readiness of CNI to weather increasingly sophisticated cyber-attacks, with many facilities believed to run on out-of-date and vulnerable IT systems."
He added: "The incident in Florida will go down as yet another near miss, but it is clear that CNI will remain a key target for hackers - inaction can no longer be tolerated. The UK government has rightly placed the resilience of CNI at the heart of its National Cyber Security Strategy in 2021."
The July 2020 Foresight Review of cyber security for the Industrial Internet of Things (IIoT) by Lloyds Register Foundation identified an "urgent need for... study into liability models, practicalities and implications for IoT markets; and exploration of potential international cooperation to develop trust in the supply chain for IIoT devices and software".
Its authors noted: "The management of cyber security risk for traditional systems already faces many challenges. These include the sheer difficulty of trying to map the complicated relationships between technical and human systems, and the challenges of communication between different communities where the frameworks for understanding risk are fundamentally different (for example, operations and board members, companies and regulators, procurement and cyber security teams)."
