A window of my house needs repairing. I know I need to do it, but it’s ended up in the ‘too difficult for right now,’ box, so I’m not rushing to sort it out. The draught is really annoying in the nine months of UK weather known as ‘grey and wet.’
I've been putting the job off for two years now—but it's costing me. Many of the CIOs reading this are likely to have their own ‘to do’ jobs in real life, and a much bigger headache in their work life: the problem of cloud lock-in.
What’s that, I hear you ask: there’s something wrong with cloud? Well, the UK’s communications sector regulator, Ofcom, thinks there is. Earlier this year, it shocked the market by announcing that it will refer the UK cloud market for investigation to the Competition and Markets Authority.
Why? Well, Ofcom identified features and practices that make it more difficult for customers to switch and use multiple cloud suppliers. They raised concerns about barriers to interoperability such as high egress fees (the charges customers pay to transfer their data out of a cloud). The regulator believes that hyperscalers set these fees at significantly higher rates than other providers. Ofcom was also concerned by technical restrictions on interoperability, as well as potential issues with so-called committed spend discounts.
See also: Amazon to UK regulator: “AWS does not charge ‘egress fees’”
I’m not going to get into a long debate on this, but I am a passionate believer in getting the full benefits of cloud. Getting these benefits shouldn’t mean you have to tear up your existing contract and start again, even if that were possible.
It may well be that cloud contract matters are out of your hands and are dealt with at a higher level in the company, or by the CFO directly. Therefore, it’s unlikely that you can just shift from one hyperscaler to another. So, part of you perhaps ruefully agrees with Ofcom’s critical assessment, but is there anything you can really do about this?
I suggest there is. A lot of developers and IT architects really do care about this and although the big contract may be signed at a higher level, it’s people like us on the ground who are accountable for making it work. Plus, it's your budget that gets pulled up for scrutiny if you end up exceeding the agreed spend.
The benefit of a technical approach
I see two ways that CIOs can push back on the way cloud is managed in their businesses. One is simply arguing your case with your account manager.
But there is also a second technical option that might end up benefiting you more, until new regulator-led cloud freedoms possibly emerge. This involves consciously working with software and platforms that don’t completely and expensively leave you reliant on just one vendor’s technology stack.
This applies to many application areas, but the one I know best is databases, so I’ll focus on that. Every hyperscaler has a bewildering variety of SQL, NoSQL, graph, and what-have-you options. These are also tiered, so you can choose a database for small-scale, mid-range, and enterprise/large scale problems.
But, you don’t want to get locked into a single level of one cloud stack alone - you want to give yourself options. The good news is you can choose ‘vanilla Postgres’ —the basic open source version of the Postgres SQL language from any of the cloud providers that you can get from postgres.org.
However, the danger with this ‘lowest common denominator’ approach is that you may run into trouble if your app grows and you need to add enterprise-level features. At this point, it might be useful to remind yourself why you wanted to work with cloud databases in the first place (and more importantly, if Ofcom and Gartner are right, and multi-cloud is where we all should aim for). You're modernising your data layer to exploit all the amazing cloud-native features you were missing, including on-prem simple scalability and resilience. If you aren’t really using those, then you're just doing a lift and shift and so, in real financial (but also technical) terms not experiencing the real benefits of cloud.
A huge part of moving your applications to the cloud, then, has to be database modernisation, but it’s not obvious how you get there without adopting a bespoke offering from one of the big cloud vendors. It would be better to choose databases that run across different public clouds and have common APIs. The cherry on top here would be working with born-in-the-cloud technologies, as, on closer inspection a lot of the products out there are just traditional technologies reimplemented on the cloud.
It’s perfectly natural to examine what your vendor has to offer. You’re in their cloud anyway, and it’s surely an advantage to consider the wide range of services and functionality that will a) work in your environment, and b) was born in the cloud.
You need to also understand that these services and functionality will always be customised in some way to the specific cloud environment. AWS’s fully managed relational database engine, Aurora, is compatible with MySQL and PostgreSQL, but it was specifically designed to run on Amazon. This means it can’t run on-prem and it certainly won’t work in GCP (Google’s cloud).
The result is that you’ll soon need to make changes to your application to fit a bespoke version of Postgres that will run very efficiently in the cloud environment of your hyperscaler partner, but reduces your overall flexibility. To some extent, you’re going to be locked into that version of the application because it can’t really work anywhere else. The same applies to the Google equivalent of vanilla Postgres (CloudSQL for Postgres), or the Microsoft version (Azure PostgreSQL).
Why it’s almost impossible to move across the cloud database ‘matrix’
Unsurprisingly, every hyperscaler is happy to offer you a flavour of ‘Postgres,’ whether you’re interested in starting at the low, middle, or high end. Think of it as a three by three matrix, where there are three low to high SQL cloud databases for each of the major cloud platforms of Amazon AWS, Google Cloud Platform, and Microsoft Azure. The bottom of each stack is fairly close to vanilla Postgres, but as soon as you scale, you are forced further and further away from that standard.
Each vendor providing three vertical offerings is great (if you want to move up vertically with them). But, what happens if you want to change cloud or add another cloud as part of a new Ofcom-inspired (or even mandated) multi-cloud strategy?
See also: Citi CTO spearheads new pan-industry ‘Common Cloud Controls’ project at FINOS
For example, how do you go from mid-range Google to low end Amazon? We talk a lot about scaling, but surely one of the main selling points of cloud is that you can go down as easily as you go up? In reality, it’s nearly impossible to move across this matrix, as there is no consistent API that works across all these products.
I’ll show this by a worked example. Let's say you develop at the RDS level on Amazon and decide for the next iteration you're going to go to Google AlloyDB in the middle. You’re hampered by lock-in at the RDS level, and you've also got to migrate across clouds, so you’re going to end up doing a lot of rewriting and managing complexity. Every time there's a migration, there’s a time and financial penalty.
This is why, under current cloud market arrangements, we can’t have nice things like multi-cloud. You can't have a combination of Aurora on-prem and in a cloud. You can't have multi-cloud with RDS; you can only do this in the vendor’s cloud environment. And if you want to scale up or down, you have to go through a process to change it.
Add in the spend discounts and it’s easier to just stick with the same cloud vendor for pretty much everything… and before you know it, you’re stuck forever. And the broken window never gets fixed.
How to achieve the advantages of multi-cloud?
The good news is that there are several open source Postgres-based databases that use common APIs, giving you portability and access to the great cloud features you want to exploit.
If you go down this path, instead of trying to navigate the complex maze of hyperscaler databases, you can keep your options open when it comes to cloud databases, and avoid some of the cloud honeytraps the vendors are dangling in front of us!
Bottom line: It will be at least 18 months (if ever) until the CMA may intervene in the UK cloud market and push to make multi-cloud as the norm, so because of the way the cloud market is currently organised you need to make database choices that save you future grief and money. Considering open source Postgres options will provide you with many of the advantages of multi-cloud.
Even better, if you start thinking about this now, you can avoid any panic move as a result of possible future multi-cloud regulation that Ofcom may impose.
You know what? I feel so inspired that today’s the day that window is finally going to get fixed.
