-- By Shobhan Lakkapragada, VMware and Gaurav Rishi, Kasten
This post is based on a joint session by the authors at VMworld 2020 (HCI2540: Data Management for Kubernetes and Modern Applications on VMware). With Project Pacific, VMware has deeply integrated Kubernetes into vSphere to enable modern application development and operations. Kasten K10, purpose-built for Kubernetes, offers critical data management capabilities including application backup and mobility. Kasten, as a data management design partner for VMware, allows enterprise Ops teams to seamlessly protect their growing Kubernetes applications in this cloud-native era.
This post explains the changes modern applications bring from a data management perspective, dives into the building blocks and expands on how Kasten K10 integrates with the VMware portfolio including vSphere, Cloud Native Storage (CNS), and Tanzu Kubernetes Grid (TKG) to provide application backup and mobility. Grab a cup of coffee and let's dive in!
Cambrian Moment for Modern Applications
We are witnessing a Cambrian moment for modern applications. IDC predicts enterprises will build 500 million apps using cloud-native tools and methods between 2019-24 (IDC Futurescape, 2018). To put that in context this is equal to the total number of apps built over the last forty years! Kubernetes is now a ubiquitous engine powering these cloud-native applications not only from an orchestration perspective but also launching an expanded ecosystem that enables organizations to build, run, and manage these cloud-native applications. As an example of this robust ecosystem, the Cloud Native Computing Foundation (CNCF) already has more than 500 member companies, and the list is still growing.
Handling State in Modern Applications
Maintaining state is an imperative across most applications - whether its keeping track of your shopping cart, health of your IoT devices, or your financial records. Traditional applications ended up using a monolithic, relational database for their stateful needs.
However, if we take a peek under the hood, we realize that the makeup of modern applications is quite different from the ones that were built before the cloud-native era. Some of the key differences include the use of smaller microservices with multiple stateful services as a part of the same application. The choice of the underlying stateful service is based on the workloads and business logic.
With microservices-based applications, polyglot persistence is a norm since every microservice picks the most appropriate stateful service for the job at hand. Some of these modern databases types include SQL Databases (e.g., MySQL, PostgreSQL, etc.), NoSQL Databases (e.g., Redis, Cassandra, MongoDB, etc.) as well as Message Queues(e.g., Kafka, RabbitMQ, etc.), and last but definitely not the least Object Stores (e.g., Amazon S3, Google Cloud Storage, Minio, etc.).
VMware Embraces Kubernetes
VMware has deeply integrated Kubernetes into vSphere and there are several exciting initiatives that will help enterprises with the development and operations of these modern applications on vSphere. With Project Pacific, developers can exercise Kubernetes-native APIs to access resources such as compute, networking, and storage. IT Operations teams can continue to use vCenter to rapidly provision and provide these resources for development and scalable operations.
While there are several Kubernetes related initiatives, another exciting update in this context is VMware Cloud Native Storage (CNS) that was introduced with vSphere v6.7 Update 3 release. CNS provides a Kubernetes storage platform across vSphere that spans block and file including vSAN. Under the hood, VMware First Class Disks (FCDs) provide block storage that is suitable for Kubernetes application lifecycles and are not tied to a specific VM instance. These initiatives help bring the constructs of handling state in a Kubernetes environment.
Kubernetes Data Management Challenges
While these building blocks provide the foundations for modern applications to handle state in this polyglot environment, we also need to think about Day 2 operational needs such as Data Management including backups and application mobility. Supporting Kubernetes application backups remains a critical need for organizations that are looking to benefit from all the agility and scale benefits that cloud-native technologies and operations provide. The Kubernetes world has different requirements and considerations, hence legacy backup solutions created to operate in hypervisor environments face some of the following challenges:
Application visibility: For modern applications on Kubernetes there is no mapping of applications to servers or VMs. Kubernetes can distribute application components across multiple servers/nodes for fault-tolerance and performance. Additionally, containers can be dynamically rescheduled or scaled on different nodes for better load balancing. This results in hypervisor-centric backup solutions not being able to protect the entire application in this distributed and dynamic environment.
Cloud-Native scale: Cloud-native applications requirements from a scale perspective have dramatically increased vs. traditional applications. Some of the underlying reasons include multiple microservices, an explosion in application components (e.g., ConfigMaps, secrets, etc.), dynamic autoscaling (clusters and applications), and polyglot persistence (viz. multiple databases used by a single cloud-native application).
Kubernetes Data Management Principles
To protect modern applications in this Kubernetes native era, a good data management solution should not only overcome the challenges listed above but also adhere to the principles listed below.
DevOps and shift left: The DevOps philosophy adopted in parallel with Kubernetes cedes control over both infrastructure and deployments to the developer (known as “shift left”). Developers define both application components and infrastructure requirements (e.g., storage or load balancers) as code. These programmatic requests are provisioned dynamically, via a CI/CD pipeline, and without an extensive change management process. In this environment we should treat the application as the operational unit. This approach balances the needs of operations and development teams in cloud-native environments. The application is what both the developer and operator ultimately care about.
Modern databases: As we outlined earlier the rise of polyglot persistence, where multiple data services (e.g., MongoDB, MySQL, and Cassandra) are used within the same application, has coincided with the growth of Kubernetes. Backups for these workloads should be integrated against Kubernetes for automated workload discovery. Workload knowledge will enable the backup solution to select the capture primitive (e.g., one or more of volume snapshots, application consistent backups, logical dumps) best suited to the application’s requirements.
Freedom of choice: A modern cloud-native backup solution needs to offer capabilities that allow for the portability of applications across clusters, regions, and even clouds with diverse infrastructure and Kubernetes environments. Kubernetes versions are typically released every quarter and additionally there are several distributions that are available from various vendors (e.g., Tanzu Kubernetes Grid (TKG) from VMware, EKS from AWS, AKS from Azure, OpenShift from Red Hat). A backup solution should allow you to restore or migrate your application and not tie it to a specific infrastructure vendor or Kubernetes version.
Solving Data Management with Kasten K10 and VMware
The Kasten K10 data management software platform has been purpose-built for Kubernetes. As a VMware design partner for data management, K10 can be seamlessly deployed with vSphere across TKG and other Kubernetes distributions. K10’s application-centric approach and deep integrations with relational and NoSQL databases, and storage systems including VMware Cloud Native Storage (CNS) provide for backup/restore and mobility of your entire Kubernetes application. K10, with operational simplicity as a core tenet, makes Kubernetes application mobility and backup as easy as 1-2-3.
Native FCD support in Kasten K10: Kasten worked closely with VMware to add native support in K10 for First Class Disks (FCDs). FCDs, also sometimes referred to as Improved Virtual Disk (IVDs), were created for Kubernetes to allow for independent lifecycle management (e.g., create or delete operations). Unlike previous vSphere virtual disks, FCDs no longer need to be associated with a VM and actions such as snapshot or backup can be performed independently on them. These FCDs can be provisioned by the new vSphere Container Storage Interface (CSI) plugin for Kubernetes. Given that FCDs are the path forward for provisioning storage for Kubernetes-based stateful applications, K10’s FCD integration allows it to connect to a vSphere instance and auto-discover all FCD's provisioned by the vSphere CSI driver in Kubernetes. Additionally, the native integration enables advanced capabilities like Change Block Tracking when creating efficient backups.
Kasten K10 integration with CNS: Another integration point includes VMware’s Cloud Native Storage (CNS) on vCenter Server. CNS implements provisioning and lifecycle operations for persistent volumes. Kasten K10 integration with CNS allows Kubernetes to perform on-demand provisioning of vSphere storage and also allows the admin to see these provisioned volumes within vCenter. Additionally, CNS interfaces with the Storage Policy Based Management to guarantee a required level of service to the disks.
Kasten K10 integration with TKG: The figure below shows the top-level constructs of the integration and the capabilities K10 provides for backup and DR operations. These include automatic discovery of applications in the cluster and automation efficiencies by using policies. These policies can be forward-looking too; so that as soon as an application is introduced into the cluster, it can be automatically protected with a predefined policy. This ensures backup compliance and coupled with the rich spectrum of backup consistency options provided by K10, VMware customers have a compelling data management solution for their Tanzu Kubernetes environments too.
Kasten K10 is also integrated and available on VMware Cloud Marketplace and you can read more about that here.
Get Started Today and Learn More
The VMware and Kasten partnership not only addresses all the data management challenges legacy solutions face but adheres to the core principles of a cloud-native backup solution we outlined earlier.
Sign up for VMworld 2020 if you haven't already and come watch the session and demos: HCI2540: Data Management for Kubernetes and Modern Applications on VMware
Lastly, here are some pointers to give this a try and learn more:
Gaurav Rishi is the VP of Product and Partnerships at Kasten by Veeam. He is at the forefront of several Kubernetes ecosystem partnerships and has been a frequent speaker and author on cloud-native innovations. He previously led Strategy and Product Management for Cisco's Cloud Media Processing business. In addition to launching multiple products and growing them to >$100M in revenues, he was also instrumental in several M&A transactions. Gaurav is a computer science graduate and has an MBA from the Wharton School.
