This year at Kubecon EU, I was fortunate enough to attend a session by Nikhil Jindal and Greg Harmon on Multi-Cluster Ingress powered by Cluster Registry. Both Nikhil and Greg work on multi-cluster Kubernetes at Google, responsible for building and maintaining tools to facilitate cross-cluster operations, such as routing. One such tool is kubemci, which enables users to create and configure loadbalancing across Kubernetes clusters through a CLI that integrates with the Google Cloud Platform API, to configure GCP load balancers. Another tool is cluster-registry, which aims to take ad-hoc config files and other homegrown tooling for interacting with multiple clusters, and replacing it with a building block with a standard API. In this talk, the pair presented a slick way to operate kubemci as a controller, to declaratively configure a GCP load balancer.
What is Single Cluster Ingress?
Ingresses are a Kubernetes abstraction for various load balancers that exist. For instance, there are on-premises load balanacers, Amazon ELB/ALB, Google Cloud Platform Loadbalancer, Digital Ocean load balancer, and the list goes on. Users simple write an Ingress spec, and an IngressController is responsible for configuring the load balancer. Once configured, users are presented with an IP address at which they can access their service.
What is Multi-Cluster Ingress?
Multi-cluster ingress follows the same principles as single cluster ingress; however, the resulting load balancer is configured to route traffc to services on multiple Kubernetes clusters. The creation of this load balancer is still driven from a single ingress spec, and still produces a single IP address for client consumption.
Why Multi-Cluster Ingress?
The most common use-case for running multiple clusters is high-availability. As you know, a zone or even entire region of your favorite cloud provider can (and will) go down, but your infrastructure should be resilient enough to continue serving requests. Ensuring applications are spread across clusters which run in multiple datacenters provides this resiliency. Serving an application across multiple clusters also increases the scalability of the application, as total resources are not bound to a single Kubernetes cluster. Another driving factor of implementing multi-cluster ingress is geoproximity to users. With an increased focus on IoT and edge computing, the ability to serve requests to customer from the nearest datacenter is paramount. In certain verticals, even a millisecond matters. Finally, users want to avoid vendor lock-in. Today, more and more enterprises are demanding a multi-cloud strategy, forcing their infrastructure to be spread across multiple clouds and on-premises datacenters. While this tends to be operationally more difficult, it mitigates risk associated with tieing oneself to a single vendor. While there are clear advantages to operating multiple clusters, it is inherently complicated to configure network resources, such as services and ingresses, across clusters.
How it works
Thankfully, the integration of cluster registry with the functionality of kubemci, running as a controller, provides a simple and declarative mechanism for routing traffic to multiple backend Kubernetes clusters. The MCI controller watches for updates to the multi-cluster ingress spec to propagate configuration changes to the global load balancer. Additions to, and removals from, the cluster list will update the load balancers routing. Users will seamlessly be routed to new clusters as they come on line, and rerouted to failover clusters if a cluster begins to fail, or is removed from the list.To demonstrate the capabilities of the MCI controller, check out the official video from Kubecon: