Apache Cassandra is a high-performance distributed database known for its fault tolerance, horizontal scalability, and ability to handle huge volumes of data with low latency. It is used by Monzo Bank to support banking services for its five million customers; by T-Mobile to underpin services for its 100+ million customers; by Netflix to handle petabytes of data which need to be reliably accessible to 200+ million users in milliseconds; and by Uber, as its application database of choice to underpin volatile workloads across its platform.
But not everybody loves it. The database is favoured for demanding, internet-facing applications running at scale, and can be reliably deployed on utility servers. For all its virtues, a reputation for cost and complexity have meant it is not yet the default database of choice for every modern data application.
That just changed dramatically, with DataStax – a heavyweight contributor to the open source database and a leading provider of distributed data management products and cloud services – this week revealing a breakthrough that it believes will dramatically lower total cost of ownership (TCO) for customers, let businesses eliminate the need for the headache of capacity planning, and importantly, allow every developer to easily and cheaply spin up their own Cassandra cluster; something that would previously have been impossible.
DataStax has announced that its database-as-a-service (DBaaS), dubbed DataStax Astra, is free to launch, with an on-demand tier based on usage and reserved capacity for larger workloads. Pick a cloud to run it on: Astra can be deployed on AWS, Azure, or Google Cloud. The secret sauce: Separation of compute from storage in Cassandra to make it serverless. The Stack joined DataStax’s Chief Product Officer Ed Anuff to hear more.
Why is making Cassandra serverless a big deal?
CIOs and people in IT spend a big part of their lives trying to answer the question ‘how many servers do I need, and what’s the most economical way to reliably get them?’ Before the cloud, when you wanted to put workloads on the internet, first you would go and put some servers in a data centre or rent some servers. Then people like Amazon came along and said, 'I can give you servers as you need them. Press this button, or do an API call, and you'll get more servers.’ In recent years we've seen that evolve into a serverless model for compute resources, where your service provider just spins up servers as necessary. If it has to launch a server in the background, it quietly does. But an awful lot of software and a lot of cloud services are still not serverless.
Sign-up to Learn More about Serverless DataStax Astra
In fact, probably 95% of what you've heard about when it comes to serverless has not been applying to data: it's been applying to compute – to where the code gets run. The database has still pretty much always needed a server and a couple of replicas of it at minimum -- because if your data goes away, you're in big trouble. But, if you can have your data in a serverless way, you can stop thinking,”‘am I going to need an extra server today? Or, “am I paying for too many servers that I don't need? In practical terms for a developer, it means your data in the cloud works in the flexible, cost-effective way you really expected the cloud to work. It’s a game-changer.
Can you give us an example?
When I speak to a major retailer, typically in October they ask, ‘what's our projection for how busy this holiday season is going to be?’ They do an economic calculus, and they scale up their servers to meet expected seasonal demand. Around late January/February, they often start shrinking their resources back down -- because if you talk to any retailer online, they do the majority of their business in the holidays.
The reason they often only scale it up and scale it down once a year is because it's a scary process: if your servers go down while scaling up or down, you're offline, and this impacts business.
As a serverless database, Astra is changing that by allowing you to scale data up and scale down dynamically, as you need it. That's pretty unique in general. But being able to do it with the most scalable database, Cassandra, and the only open source and multicloud serverless database - one that you don't get from one of the big three cloud vendors - is pretty significant.
Cassandra can handle massive levels of scale. A lot of big retailers like Netflix and Apple, have thousands and thousands of Cassandra servers running. DataStax customers like T-Mobile, Home Depot and countless others -- have a whole bunch of servers running all the time. The reason they chose Cassandra is typically because it can handle global scale; it literally is ‘thousands of interactions driving thousands of transactions’ per second. In a lot of these use cases the scale can be tremendous: the swings can save companies millions of dollars.
With DataStax Astra now serverless, you can auto scale based on traffic and app requirements, scale down to zero automatically when the database is not in use, and deploy apps that can scale infinitely from day one. We estimate the TCO savings to be three-to-five fold using Astra compared to provisioning for peak capacity for the same workload deployed on-premises. That captures some of the benefits of this breakthrough at the top-end of the user base. But lots of people say 'if I get that big, I'll switch to Cassandra,’ because Cassandra is great for running large workloads. But before we introduced the serverless capability, it could be a little complicated for running smaller workloads. With Astra serverless, it’s easy for developers to deploy Cassandra faster, and in a much more lightweight way: they can avoid infrastructure configuration changes, eliminate capacity planning, and get rid of any guesswork on storage and compute needs of future apps.
We are bringing Cassandra through as an option for more developers to start with, rather than something they turn to for scale later.
A breakthrough in separating storage and compute is in part to thank here, right?
Right. Most databases have their own bespoke file management system that handles the data because the database needs to keep the data persistent -- so it keeps it on the hard drive, or the SSD, and it is constantly controlling all of the access to that storage. So if you're doing a complex query for analytics, or you want to go and only update certain users based on certain conditions, all of this typically happens in a monolithic stack because there's a whole bunch of things that need to happen at one time. If I'm updating a credit card transaction, for example, it absolutely must complete correctly and predictably.
Traditional databases typically achieve this by running on a single server. If I have to do more transactions, if my website is getting more traffic, I ‘scale up’: i.e. I get a larger server. If you want to run a database, even in the cloud, this again becomes a question of how big a server -- or a node, or whatever terminology -- do I need to handle this data? Separating the storage from compute is a game changer, as you can scale compute and data up or down independently from each other. Cassandra is uniquely suited to this type of world, because – unlike some databases – you can already scale its compute horizontally as much as possible.
Before serverless Cassandra, users would typically scale Cassandra clusters for two reasons: They would say either, 'Okay, I need to serve more requests, meaning I need to scale my compute: I better add some servers.’ Or my requests might not be scaling, but my data might be scaling. Regardless, I would still have to add servers, because we would scale both the compute and the storage together.
That gave you the power of Cassandra to handle scale. But this was quite inefficient: for example you might have situations where your data is growing and the requests aren't changing very much and you're left thinking ‘why do I have 500 servers just have this data?’ You can forget all those inefficiencies with DataStax Astra.
You launched a beta in December. Are you confident in the final product?
Absolutely.
It’s required some innovations within Cassandra to figure this out: we had to decouple the storage layer; we had to make it pluggable; we had to make it talk to block storage; we had to make it expect that the storage was accessed via APIs, we had to figure out what was necessary to make it talk to block storage.
Once we did that, we got into all of the issues around now these things are decoupled, how do we go and build the algorithms for going and scaling up this compute as necessary. It’s critical with a database that nothing goes wrong. You need to do lots and lots of testing to figure out all of the edge cases.
The other thing that took a bit of time to figure out was how to make all these improvements in as modular a way as possible, because we want to make these available to open source users and people using Datastax on-prem later this year. Now, the first thing that developers see with Astra serverless is that they can go in and get started right away. We can give you as many databases as you need. You want to have one database for production and one database for testing? Maybe one for a third because you're working on the next version of your application? You can just do it: it’s simple and it starts up right away. Your database experience is dramatically different. Then the bill at the end of a month as a developer can be as little as $10.
You can see we're at an inflection point that completely changes the way you can think about data -- and we're doing it with a database that makes this super interesting, because there is really no effective upper bound. Which means that whether you're an enterprise and you're doing a new project, or you're a startup, or an internet company, you can afford to be more experimental.
The amount of agility and freedom that it gives people is pretty significant.
This Q&A with The Stack was sponsored by DataStax.
