Amazon, Google, and Microsoft have teamed up on a project called Kro that is focused on “making K8s APIs simpler for all Kubernetes users.”
The unlikely bedfellows say that “Kubernetes lacks a native way for platform teams to create custom groupings of resources for consumption by end users” – and that Kro might help users avoid custom toolsets.
This is handled in many organisations by Helm, but as Wilson Spearman, CEO of Parity blogged: “Helm’s architecture has revealed fundamental constraints in its ability to properly manage dependencies, handle [custom resource] upgrades elegantly, and properly manage lifecycles.
“No matter how much effort Platform and DevOps teams spent creating clean Helm charts for their end users to self-service k8s, they often ended up spending their days wrangling cluster after cluster regardless…”
Enter a baby Kro...
Kro, short for Kube Resource Orchestrator, is not yet ready for production use, its team of contributors warns – but has garnered growing interest.
(Kubernetes, aka K8s, has become the standard for managing containerised applications’ resources. As Datastax’s Patrick McFadin has put it: “Kubernetes is providing the ability to use clouds as a commodity for compute, network and storage. We can create entire virtual data centers and choose where we run them.” Users can “copy and paste” databases or other software to any environment that supports K8s.)
Last week Kro kicked off its first bi-weekly community meeting (with a large contingent from AWS, slightly smaller ones from Microsoft, Google, Spotify, Swisscom, and Adobe, among others represented in the room.)
Why "Kro"?
As Microsoft's Bridget Kromhout puts it:
- "Kubernetes-native: kro extends Kubernetes via Custom Resource Definitions (CRDs), so it works with any Kubernetes resource and with your existing tooling.
- "Approachable end-user experience: kro simplifies defining end-user interfaces for complex groups of Kubernetes resources, making it easier for people who are not Kubernetes experts to consume services built on Kubernetes.
- "Consistency for application teams: kro templates can be reused across different projects and environments, promoting standardization and reducing duplication of effort (while making life easier for platform engineering teams!)
Swiss telco Swisscom’s attendees said that they are looking at Kro as potentially a way to help manage their containerised 5G core resources.
As host Jesse Butler, from the Kubernetes team at AWS said during the February 19 meeting: “All three of the CSPs [cloud service providers] are here; we all were working on this problem and… just decided rather than to let lots of chaos happen over a year or two and then finally, probably, find a standard and come together, we just decided to work together!”
Kro’s GitHub notes that “building with Kubernetes means dealing with resources that need to operate and work together, and orchestrating this can get complex and difficult at scale…we're taking a step in reducing the complexity of resource dependency management and customization.
Enter ResourceGraphDefinitions (RGDs)
Bridget Kromhout, a Principal Product Manager at Microsoft Azure, explains that Kro “introduces the concept of a ResourceGraphDefinition (RGD), which specifies how a standard Kubernetes Custom Resource Definition (CRD) should be expanded into a set of Kubernetes resources.”
(CRDs are an extension of the Kubernetes API that is not necessarily available in a default Kubernetes installation. They also let K8s users create new types of resources without adding another API server…)
“Users specify their resources and the dependencies between them in a RGD, other users can deploy an instance of that definition, and kro will dynamically manage that instance, including propagating updates and driving the collection of resources to their goal state” as she wrote.
One big early challenge for the project is optimising quite how much those RGDs do and avoiding creating bottlenecks or complex caches.
Early questions being worked on...
As Amine Hilaly, AWS, wondered in the meeting: “What's the maximum reasonable amount of resources we should watch in a cluster? I don't know if Kro should be watching all the resources or not…
"I think Argo already watches all the states of the cluster. We've seen a lot of scale issues when it comes to watching everything and caching everything in memory… we should discuss whether Kro should be watching external resources; today Kro doesn't do that.”
Microsoft’s Matthew Christopher added, as the team started to work through some of these early “mega” issues: “I do think that there's a way that we could say we only watch what you put – like the names of resources that you put in RGDs and nothing else; that maybe dodges some of the Argo scale issues” he said. (Argo, which reads environment configurations and deploys them automatically to a Kubernetes cluster, has had recognised issues with scaling and memory cache-related issues.
AWS’s Jesse Butler added in the February 19 meeting “We [also] need to make a decision on how changes are propagated in instances of resources when RGD changes are made and then how configurable is that? Like we'll have an opinionated default but what's the right approach?”
It’s clearly early days for Kro, but with a collaborative, welcoming community with some bright minds from many corners of the Kubernetes cosmos, clearly very much one to watch in coming months and years.
The project's GitHub is here and it also has a handy Slack channel.
