A cyber incident at a critical supplier to Vodafone had “potential scope to impact the entire telecommunications industry” according to a stark warning by the British multinational.
“The frequency of such incidents [at suppliers] is increasing… we contractually require our suppliers to report incidents and we manage these incidents as if they were internal” Vodafone said.
The supplier in question “manages the netting of roaming charges between operators and reported a cyber incident in September 2021" the company said in its latest annual report.
Vodafone declined to name the supplier when approached by The Stack.
We assess with a high degree of confidence that Vodafone was referring to Syniverse, which in September 2021 reported a breach that had gone undetected by the supplier for five years.
(The company, which provides wholesale roaming and other services to a global network of telecommunications companies, names Vodafone as a "satisfied customer" on its website.)
Syniverse hack missed for five years, reported by customer
The Syniverse hack was first reported by Vice’s Lorenzo Franceschi-Bicchierai in early October 2021 after an SEC filing revealed that in May 2021 it had become "aware of unauthorized access to its operational and information technology systems by an unknown individual or organization" that began in May 2016.
“Syniverse has notified all affected customers of this unauthorized access where contractually required” it said at the time (our italics: a reminder if any were needed of how critical contractually obliging such disclosure by partners is) and adding that it saw no attempts to disrupt customers or monetise the breach.
Syniverse provides real-time charging and billing, data optimisation and other interoperability functions including data clearing, financial clearing and fraud protection as well as global messaging interconnectivity between mobile network operators. The company processes more than 740 billion text messages every year and has "direct connections” to over 300 mobile operators globally, including hundreds of blue chip telcos.
In previously unseen update, Syniverse shares more details
In a previously unreported update on the Syniverse hack seen by The Stack, the company admitted to regulators that it was only alerted to the compromise of its network after "a customer contacted Syniverse regarding unauthorized activity that appeared to be originating from Syniverse’s Electronic Data Transfer (“EDT”) environment" (a file transfer platform used to exchange roaming records between Syniverse and its customers).
"Syniverse’s EDT environment and several discrete databases within Syniverse’s network... were accessed by the unknown individual or organization" it added in a January 2022 SEC filing, describing the EDT environment as "a file transfer platform that is used to facilitate the exchange of roaming records between Syniverse and its customers so that Syniverse can provide clearinghouse and other roaming services."
* Subscribe to our > Command Line newsletter < on LinkedIn *
"Login information allowing access to and from the EDT environment had been compromised for approximately 235 Syniverse customers" the company said, saying the attacker had also attempted to access a limited number of Syniverse databases. One of the databases that was accessed contained GSM registration data for a 24-hour period. This data included IMSIs (international mobile subscriber identification) and MSISDNs (mobile phone numbers) but did not include subscribers’ names or other information identifiable to any individual or device".
Vodafone supplier hack: Only "minor direct impact" says telco
Vodafone said it had suffered only "minor direct impact" from the supplier breach in its annual report, declining to confirm that it was referring to Syniverse (spokesperson: "I am not going to speculate about the vendor") when approached by The Stack but telling us an investigation “found no evidence to suggest Vodafone customer data loss" and adding in an emailed comment that "protecting our customer data is our highest priority. We have an international team of cyber security professionals who protect, defend and monitor our systems.”
Syniverse says investigating and responding to the incident has cost it $4.7 million; its cyber insurance will cover $3.7 million of this. The company has since "substantially increased" logging, visibility, and monitoring in order to detect and remediate any further unauthorised activity. (A subsequent attack bearing "tactical similarities" to the earlier breach saw an unknown hacker gain fresh access Syniverse’s network but its new monitoring systems "detected this activity and it was terminated prior to access to any sensitive information or assets.")
Among other measures to strengthen its cybersecurity it has "deployed end point detection and response (EDR) technologies extensively throughout its network... conducted an in-depth assessment of its entire server network and has a plan in place to address any servers incompatible with current EDR technologies by either decommissioning them or upgrading them"; removed "known backdoors that would allow access to Syniverse’s network without proper credentials" and "reset, recertified or inactivated all credentials."
In telcos, the "scream test" isn't an option...
Security consultant James Bore, who spent over two years building and running operational security for a major UK telco, told The Stack: "People underestimate the complexity of telcos, assuming that everything's a homogenous environment when you're usually talking (at least for the infrastructure providers rather than MVNOs) about networking and computing infrastructure grown on a national scale over decades with massive diversity of technologies and supportability. Anything used as a national utility (in effect) which has been running for more than a decade is going to be largely made up of technical debt, with critical systems provided by everything from the very latest cutting edge tech, to mainframes which are irreplaceable without rebuilding everything around them. This means visibility is almost non-existent, institutional memory relies more on people than on oversight systems carrying out network discovery, and everything that is unknown could well be critical.
"The scream test doesn't work, because turning off the wrong server doesn't only have a business owner screaming, but through complex and undocumented interwoven webs of interdependencies may knock out communications for a region, or the country. And of course as a telco you're reliant on a number of upstream providers, shared by almost every other telco. The Syniverse example is one, and a few years ago Ericsson's failure to simply renew an SSL cert took down telcos across 11 countries for hours. It only takes one mistake, or in Syniverse's case failing to look in the right place to detect, and an incident can go on for years..." he added.
Whilst the Vodafone supplier hack draws attention amid a rise in supply chain incidents for the cited "potential scope to impact the entire telecommunications industry” it was another attack on its Portuguese network earlier this year that ultimately had more of an impact. In that incident 4.7 million mobile customers lost services for eight hours after a network outage caused by "a deliberate cyber attack that was intended to cause disruption."
According to Vodafone's annual report, no malware or malicious software was installed during that incident and the attack method "would be described as a 'living off the land' attack because it "did not use any specialist tools... the attack relied on sophisticated social engineering and a deep understanding of IT systems and networks". No customer data was lost in the attack, which saw some services affected for up to 48 hours.
