Rethinking the UK's "P-8" surveillance plane data challenges

The cockpit of the Boeing P-8 Poseidon is a busy space. So, often, are the seas below it when the maritime surveillance plane is out on a sortie. The Royal Air Force’s nine planes track submarines, keep a watchful eye on critical subsea infrastructure, and even get involved in search and rescue.

Bar the lack of windows, from the outside the P8 looks much like the Boeing 737 airliner, with which it shares an airframe. The telltale bumps and bulges across its body do not shout “surveillance equipment” quite as loudly as the E-7 Wedgetail’s spinal “wedge” does, but the P-8s sitting on the runway at RAF Lossiemouth in Scotland are riddled with equipment. 

The planes feature a wide range of electronic sensors, radar, a high-fidelity camera and can carry 129 “sonobuoys” – disposable, floating widgets about a foot wide that dip sensors below the sea surface. These are shot from the plane by a pneumatic launcher from L3Harris that is capable of ejecting one of these aquatic listeners every three seconds.

As Peter Armitage, the officer in command of the P-8 tactical operations centre puts it: “Let’s make no bones about it: Our job, as P-8, is to protect the continuously-at-sea nuclear deterrent the UK has. We do that by building recognised maritime pictures of surface and subsurface; knowing where the people are that we would not like to be within arm's reach…”

The teams operating the P-8s work out of a newly modernised RAF Lossiemouth; the northern base for the UK’s Quick Reaction Alert since 2014 and home to Typhoon crews that are on 24/7 alert to be scrambled to potential threats in the UK’s airspace. (One recent sortie saw a P-8 from Lossiemouth out monitoring a Russian task group including destroyer RFN Severomorsk, landing ship RFN Alexander Shabalin and two transport ships that were returning from the Syrian coast via the English Channel.)

Big blue sea, big piles of data…

Every sortie by the planes generates terabytes of unstructured data. Analysing that data and sharing it for prompt analysis has been challenging. When the planes were commissioned it was a highly manual task involving lifting storage arrays of the aircraft and having an RAF team laboriously label vast quantities of metadata in a locked down location. 

(Picture, if you like, someone scrolling manually through hours of tape showing mostly just the deep blue sea, before they come to the motherload of submarine signals that they actually need to analyse.)

That was a challenge not just in terms of time-to-analysis, but made the data distinctly siloed at a time when there is demand from other intelligence and military stakeholders for such vital intelligence and interoperability with naval assets and their own intelligence is vital.

The RAF airbase that the P-8s are based in recently saw a £350 million overhaul. With an eye to tackling these challenges, their data infrastructure has been modernised too. On a recent press trip attended by The Stack, the Ministry of Defence (MOD)’s “Defence Digital” organisation and their data partner for the P-8s, NetApp, walked a small group of journalists through how the RAF is working towards making sortie data more easily discoverable, interoperable and resilient.

The new setup involves the deployment of NetApp E-Series storage arrays on the P-8s and a new “data fabric” underpinned by a Fortinet-backed SD-WAN network that enables automatic backups to a second site outside Lossiemouth, wider terminal access for analysts and critically, automated metadata tagging that lets users find the needle in the haystack far faster (the time to analysis has gone from “weeks to minutes” says the RAF.) Even better, Navy analysts can also upload and analyse data to this mesh.

Defence Digital’s Toby Milwright, “Poseidon Air ISR LDO manager” emphasises that a key focus for Defence Digital is ensuring greater interoperability with other partners (including fellow P-8 owners Australia, Norway, the US, New Zealand, and soon Canada and Germany) and other defence stakeholders, although he freely admits that there is work to do still: “Multi-domain integration is the grand strategy [but] joining secure networks is not an easy task…” he says in a short interview.

“Platforms like Poseidon need to be integrated with that UK network so they can work with the rest of Defence. It's the same for other sorts of platforms. We’re really majoring now on that sort of integration piece…”

We’re not, he adds without going into detail, “just talking about data and communications, but the ability for the people to work together… there are lots of angles to that; information and data is one of the upper layers  that directly relates to sort of operations and integration. But there's lots of engineering stuff at a lower level that has to be thought back to…”

On that data layer, MOD partner NetApp (which arranged the trip) is happy to offer a touch more detail. The company’s Ewen Litster explains that its StorageGRID proposition which the P-8s operators now use is object storage that “utilises S3 protocols, and it's in a data mesh,” he says.

“So you'll have local storage, secondary data storage for resiliency. We also have local storage for the end users from a production point of view… the naval element will share some of their data from their sensors and their capabilities into the system, and that can build a bigger operational picture. The ability to have local storage, but actually access any of the data across the entire data fabric is [one of the] real advantages.

“There are resilient controllers in the system, and you have load balancer capabilities as well to separate the data across all of the nodes. 

“We use erasure coding, which is 2+1 on this particular system, so there's three copies of every every bit of data, so there's resiliency across all of those storage platforms within the mesh,” NetApp’s Litser adds.

In response to a question from The Stack, he adds: “Our policies manage the data movement but yes [Fortinet] manage the bearer capabilities… 

“[MOD are] agnostic as to what's available to them [in terms of networks]… they can utilize satellite communications. They can utilize what we term in the military ‘dirty internet’, so localized internet connections. But they also utilize their own Defense Digital infrastructure and they use encryption devices to enable and manage that,” he adds.

Defence Digital's Milright adds: "[StorageGRID is] the big hub of data that's safe, secure, it's resilient, and then it can be shared from there. [I spoke earlier about] Waddington, which is the other RAF base [and] there are other points where we share the data securely over Ministry of Defence networks. Then we get into sort of, how can we develop this and take this forward, and perhaps look at sort of deployed versions as well..."

The RAF’s Armitage notes in a presentation that data dissemination and interoperability are key to the success of the P-8 programme. 

Critics have grumbled that nine of the aircraft is not nearly enough for the UK: as former Air Vice-Marshal Andrew L Roberts put it in 2023, nine of these planes are “insufficient to guarantee adequate maritime patrol aircraft (MPA) support for UK operations in the Indo-Pacific region whilst simultaneously supporting other essential tasks such as providing continuous cover for the UK strategic deterrent. If this situation is to be corrected, up to seven additional P-8s (depending on RN operating areas) would need to be acquired, bringing the total fleet to 16 aircraft” he said.

Staff at RAF Lossiemouth are hardly about to get into the politics of this but they do want to emphasise that they're getting some more "Wedgetail" surveillance aircraft later this summer and as Armitage puts it, given close relations with allies “the key message here is not that the UK has only got nine, it is that we've got 40 aircraft in Europe that can deal with whatever threats we need to deal with… [we also have] this program called JEDI – Joint Exploitation and Discussion of Initiatives," he adds.

"It's basically a key interoperability forum where we talk to our allied nations to see how we work better together; where we can discuss tactics, where we can discuss how we operate together. How do we co-crew? How do we engineer?”

See also: UK defence facing a “step-change” in how adversaries use digital and data: “Attitudes, culture” must change

So interoperability and partnerships are key here, in conclusion; across physical assets and information technology; across allied partners and other  stakeholders in defence and the broader intelligence community. 

Listening closely and thinking of the the intelligence lifecycle of “collection, processing, analysis, and dissemination” then headway is being made on the first three and the last is the toughest nut to crack. 

So is forward-planning on such programmes and future-proofing them.

MOD’s Milright notes: “In Defence Digital there are programs for satellite communications, cyber defense [etc] It's a bit of a matrix management job; you get a programme coming in from the side saying, ‘can I have a bit of everything, please? And by the way, I'd like to do it my own way…’

“This is a US delivered aircraft designed to work with US networks. “[We’ve had to make it] work with UK networks. So that's an integration problem of network levels. And in security terms, we have very similar rules, but not quite the same rules, so that that starts to become interesting, and then we get into the data that's coming off [the P-8]

“How do we in the UK, really want to use it? We'll want to use it collaboratively, but we also want to be able to use it within our own context – and that's with defense as a whole, Defense Intelligence, other UK defense agencies and intelligence agencies, as well as working within NATO, so that data problem starts to become more important…

“[We want to be] enabling those communications between aircraft and ship, between aircraft and UK command centers or deployed command centers, and how… actually make sure that that works securely. 

“And then finally, this is a constantly shifting theme: there are strategic policies in defense, strict strategic policies in Defence Digital about where we're going, and that doesn't necessarily align with an aircraft platform program, and we have to find ways to join the dots – so there so plenty of challenges in the job.” With a fleet of E-7 “Wedgetail” aircraft set to join the P-8s at Lossiemouth sometime later this year, that challenge will only continue. Does NetApp have a role in this latter programme? The company’s reluctant to give a straightforward “yes” during a briefing but as a spokesman says, “there’s a futures picture within this programme… [with regard to whether] other elements will be integrated or expected to be integrated in the future? Yes. So NetApp StorageGRID will be the underlying infrastructure that will enable that Data Fabric,” he concludes.