This article was co-authored by Tom Manville, VP Engineering Kasten by Veeam, and based on a community white paper from the Kubernetes Data Protection Working Group.
As Kubernetes takes over modern infrastructure, I find it useful to consider its more humble beginnings. When Kasten started in 2017, the number of people trusting their data to Kubernetes applications was relatively small. The market consisted of early adopters. Even if they had the desire to run things like databases in Kubernetes, much of the tooling and automation did not exist at a level required for production data services. Support for data protection was not a topic that was often discussed when we entered the market.
Fast-forward to the present day. The community, and specifically the Kubernetes Data Protection Working group, has put together this white paper that discusses the support Kubernetes provides out-of-the box for data protection. I find it extremely impressive how much progress the community has made. I hope this paper is able to adequately convey the amount of work done by so many people. Keep in mind that this is only the tip of the iceberg.
I’m writing this blog post to share Kasten’s perspective as the leader in this space. (If you don’t believe me, see what the analysts say.) Kasten has been active in the Data Protection Working Group from the beginning and was one of the companies present at the initial formation of the Working Group. Kasten has been working to bring the lessons we’ve learned as a leader in Kubernetes Data Protection back to the community through this channel and through our open source efforts, including Kanister.
One of the key takeaways from the white paper is that data protection for Kubernetes is actually a necessity. At one point in time, all Kubernetes applications were truly stateless - if the application stopped working or its Kubernetes state got corrupted, it could be terminated and restarted without any side effects. As Kubernetes applications have evolved, stateful applications have become more common, and loss of the application state is not acceptable in many cases. One common use case has state stored in Persistent Volumes (PVs), and it’s easy to understand when the state in those volumes needs to be backed up and restored. As the paper notes and as we’ve observed, even if a Kubernetes application doesn’t use PVs, it may still have state in Kubernetes resources that needs to be protected.
The white paper calls out a number of use cases that need to be solved today. Let’s take a look at how Kasten K10 handles these use cases.
How Kasten K10 Handles Backup, Recovery and Mobility in Kubernetes
This diagram from the white paper shows workflow for Kubernetes backup and identifies the various building blocks, some provided by Kubernetes currently and others which have been identified but not added to Kubernetes yet.
Figure 1: Backup Workflow with Missing Building Blocks (Source: Data Protection Workflows White Paper)
Kasten K10 includes all of the components needed for this and other workflows, using Kubernetes standard pieces where available, along with Kasten K10 provided components and Open Source components.
A key area that is called out in the white paper and the diagram above is protection of applications and how to define applications. We at Kasten feel this is a key distinction; modern applications are composed of multiple components – the entire application needs to be protected, and the protection needs to be orchestrated together. Kasten K10 is designed to think in terms of applications. There are many ways Kubernetes applications can be defined by developers, and it’s important to be able to back up and restore all the different variations.
Kasten K10 is able to handle the majority of applications by understanding the common deployment patterns in Kubernetes. For more complex applications, Kasten K10 provides the ability to customize the execution of backup and restore operations through intuitive, Kubernetes-based APIs. In some cases, application backups may require specific protocols in addition to the standard Kubernetes lifecycle controls. Any Kasten K10 operation may be extended and customized. The white paper includes several examples of dataservice-specific commands that are needed during data protection operations.
Disaster recovery is a critical scenario for data protection. Kasten K10 is able to handle all of the scenarios outlined in the white paper: recovering to the same namespace in the same cluster, recovering to an alternate namespace in the same cluster, recovering to an alternate cluster in the same region, and recovering to an alternate region or geography. Kasten K10 handles this by exporting snapshots of the applications, including Kubernetes metadata and volume data, to an object store such as AWS S3 including S3-compatible object stores, Google Cloud Storage, and Azure Storage, as well as the Veeam Backup Repository. Kasten K10 uses the Open Source project Kopia to manage data in object stores and is a major contributor to the Kopia project.
A key part of a cross-cloud strategy is the ability to move applications between clouds as well as within a cloud. Kubernetes provides a largely cloud-neutral environment across multiple cloud providers, and Kasten K10 supports the migration of applications between clusters, including cross-region and cross-cloud.
Another important use case discussed in the white paper is the ability to recover portions of an application. Kasten K10 provides the ability to recover selected pieces of an application backup, such as a specific volume or a specific Kubernetes resource. Kasten K10 also enables data-only restore, which can be used to roll back an application’s state to a previous point in time.
One area that is not touched on in the white paper is ease of use. Kubernetes is designed to make many things possible, but that doesn’t always mean it is easy to use. Kasten K10 provides an industry-leading UI as well as Kubernetes-native APIs for control of backup, restore and scheduling. Data protection is a complex task, and Kasten has made it a priority to not only create an enterprise-grade solution but make that solution usable without large investments in training.
Committed to Improving Data Protection for Kubernetes
In addition to handling all of the use cases outlined in the whitepaper, Kasten has committed resources to improving data protection on Kubernetes. We are founding members of the Kubernetes Data Protection Working Group and contributors to the white paper. Kasten is actively involved in improving data protection on Kubernetes by working on the missing building blocks identified, including backup repositories, changed block tracking, application snapshots, and other Kubernetes features that need to be defined and added to the platform. Kasten is contributing code to the open source community with projects such as Kanister and Kubestr, and our involvement in the Kopia project.
We’re very excited about the publication of the Data Protection Workflows white paper by the Data Protection Working Group. It’s a natural outgrowth of the increase of data stored in Kubernetes and the maturation of Kubernetes applications as important and critical resources. We invite you to read the paper and see how the scenarios apply to your own environment and needs, then investigate solutions for those needs, especially Kasten K10.
The Working Group welcomes more participation, and we encourage anyone who is a user of Kubernetes with data protection needs, a developer of Kubernetes-based applications, a data protection/storage vendor for Kubernetes and others who have interest in the area to join us. The roadmap is still evolving, and there is opportunity to extend it and add needs, use cases and potential technologies. You can learn more about the Data Protection Working Group here: https://github.com/kubernetes/community/tree/master/wg-data-protection
Data Protection Workflows White Paper
GigaOm Radar for Kubernetes Data Protection