Comparing Three Hosted Kubernetes Platforms

Google’s original brainchild, Kubernetes, has become indispensable across the tech landscape. According to a 2020 Cloud Native Computing Foundation (CNCF) survey, 78% of container users use Kubernetes. That explosion of popularity has birthed over 109 external tools to date. Keeping up with all of the latest tools is not only a challenge, it’s also incredibly time consuming. That’s why, In this article, we’ll help you answer an important question for those just getting started with hosted kubernetes: which hosting provider you should use, and why?

If you already have some context for Google, Azure, and AWS cloud services, feel free to jump to Which Hosting Provider Is Right for Me? to review a series of comparison tables.

The Case for Vendor Hosting

Provided you can foot the bill, external hosting can jumpstart your Kubernetes deployment. Depending on the type of hosted service you choose, your team won’t have to worry about the following:

• Manually provisioning the control plane components such as etcd and kube-controller-manager
• Implementing role-based access control to secure your separate environments
• Manually adding nodes to expand your cluster
• Manually configuring the pods that support your containers
• Instrumenting monitoring and logging for your cluster

While Kubernetes is fairly intuitive and powerful, its functionality has an intangible cost: a learning curve. You’ll need to spend many hours learning Kubernetes and its mechanisms.

Hosting providers remove some of this burden from your shoulders, as autoscaling, security updates, infrastructure monitoring, and maintenance come standard.

Now let’s evaluate the three leading Kubernetes vendors.

Which Hosting Provider Is Right for Me?

Deploying Kubernetes via hosting providers can save plenty of time and energy. Your decision will depend on a number of factors, such as familiarity, technology stack preferences, cost, and infrastructure priorities. We’ve put together the following tables to help you decide what’s best for your cloud computing needs.

Following the tabular comparisons, we provide a separate review for each hosting provider.

Google Kubernetes Engine (GKE)

Google first released Kubernetes before handing the Cloud Native Computing Foundation (CNCF) the reins in 2015. GKE is also the oldest service on our list. It’s little surprise that Google thus provides such robust hosting functionality—buoyed by its focus on scaling and templating.

Compatibility

Google Kubernetes Engine supports the following K8s versions:

• 1.16
• 1.17 (stable channel)
• 1.18 (regular and rapid channels)
• 1.19 (rapid channel)

Google regularly releases version updates, with three January releases (2021-R1, R2, R3) having hit servers by the time of writing. GKE allows teams to be conservative, on the bleeding edge, or somewhere in between—based on risk tolerance and stability requirements.

Over a dozen total variants of each minor version are available, making GKE unparalleled in terms of flexibility. Google recommends that you enable auto-upgrade for clusters and nodes. However, you’re free to manually select a Kubernetes version for your clusters.

Similarly, OS support is as follows:

• Container-Optimized OS (COS). Default, Google-maintained Linux variant that supports Docker containers
• Ubuntu
• Windows Server 2019
• Windows Server 1909

Docker is the default container runtime for Google Kubernetes Engine. However, both containerd and gVisor (Google’s proprietary, Linux sandboxed architecture) are supported. This makes GKE the most inclusive service for container runtimes, over options like Amazon Elastic Container Service (EKS) and Azure Kubernetes Service (AKS).

Feature Highlights

Even simple applications can be tricky to deploy in Kubernetes. GKE circumvents this traditional challenge via templating—providing “single-click clusters” and open-source applications via Google Cloud Marketplace. These provide necessary licensing and portability out of the box, for easier ramp up.

Because these pre-made applications are Google-built, they’re thoroughly tested and trustworthy for commercial use. They may be stateful (requiring storage for client data retention), or serverless (via Google Cloud Run). Additionally, an integrated CI/CD pipeline allows for swift application development while following best practices. These processes promote testing and security galvanization.

Let’s talk autoscaling. Not every vendor does this well, but GKE’s approach is both horizontal and vertical. Horizontal entails pod scaling via CPU utilization. If pods are straining available compute resources, GKE will kill those pods (or replace them) to reclaim performance. How many containers run per pod (or group of pods) will influence this behavior. This is because pods don’t self-heal, or survive, when resources become scarce.

Vertically, GKE monitors pod CPU and memory usage while dynamically making resource requests. This prevents applications from seizing or causing problems. Node pools and clusters amongst them are scaled as workloads change.

Lastly, Google Kubernetes Engine offers a variety of zonal clusters and regional clusters. A Kubernetes control plane oversees both single-zone and multi-zone clusters—though the latter employs a replica running in a single zone, alongside nodes spread across zones.

Regional clusters are uniquely beneficial. They maintain superior uptime (99.95% vs. 99.5%) over zonal clusters, and leverage multiple control pane replicas. This is better for dynamic node pools. However, they consume more resources.

Monitoring and Logging

Container performance logging is optional and opt-in. GKE routes all container metrics to Google Cloud operations suite—Google’s equivalent to Amazon CloudWatch. Formerly dubbed Stackdriver, the service also collects traces in addition to providing real-time metrics visualizations. Threshold-based alerts and other notifications are available.

Node auto-repair is enabled by default. This sets Google’s offering apart from Azure. Control plane logging is optional and off by default.

Pricing & Support

GKE costs $0.10 per hour, per cluster—with a maximum of 50 clusters per zone and per region. Accounts with a service-level agreement (SLA) benefit from an uptick in uptime. One zonal cluster per account comes free. Management fees are nil until your account exceeds 720 zonal cluster hours per month.

All GKE accounts benefit from Google’s dedicated Site Reliability Engineers. These professionals provide support and advice for maximizing application performance, or tackling runtime errors.

Azure Kubernetes Service (AKS)

Microsoft released its open-source AKS in 2018, and the service has since steadily risen in popularity. Like GKE, Azure Kubernetes Service aims to help teams launch applications quicker via baked in CI/CD support.

Compatibility

Azure Kubernetes Service supports the following K8s versions:

• 1.17
• 1.18
• 1.19

Microsoft expects to deprecate Kubernetes v1.16 in January 2021. Additionally, Kubernetes 1.20 enters into AKS preview this month—graduating to General Availability around March 2021.

Azure Kubernetes Service only supports two subversions of each minor version at any given time. That means 1.17a, 1.17b, and so on for each Kubernetes version listed, according to Microsoft’s documentation. The current version of AKS is v0.59.0 at the time of writing.

AKS also supports the following OSes:

• Ubuntu 18.04 (cluster K8s versions 1.18.8+)
• Ubuntu 16.04 (cluster K8s versions pre-dating 1.18)
• Windows Server 2019

Docker remains the default container runtime for AKS. Additionally, you may choose containerd instead, should it better support your usage. The latter works as either a low-level runtime (via cri-containerd and fulfilling the OCI specification) or high-level runtime using Kubernetes Runtime Class.

Containerd users can completely forego Docker and dockershim alike. This benefit isn’t exclusive to AKS containerd users. However, AKS is the only vendor offering a container runtime dichotomy.

Feature Highlights

AKS champions rapid application development. It firstly accomplishes this via first-party integrations. AKS interfaces with Visual Studio Code Kubernetes tools, which facilitates (in part) the following:

• Exploratory tree views of clusters, including views of workloads, services, pods, and nodes
• Access to Helm repos, chart creation, scaffolding, and snippets
• Building and running of Dockerfile-based containers
• Running of commands and shells within application pods
• Easy bootstrapping of applications—including bugging and deployment
• Live resource monitoring via cluster explorer

This is available for free from the Visual Studio Marketplace. Installation is easy, provided dependencies like kubectl and docker or buildah exist on your machine.

Additionally, Azure Monitor integration provides access to continual telemetry data—including performance, health, and resource allocation. Azure DevOps works to supercharge the CI/CD process. Canary deployments are rapid, tracing provides failure detection, and debugging is relatively simple.

Cluster monitoring is based on Prometheus by default. This relies on queries, dimensional data, and visualizations. The latter leverages a built-in browser, console template language, or Grafana integrations which add meaning beyond simple numbers.

Elastic provisioning lets you scale multiple container instances without server management. Like GKE, AKS includes event-driven autoscaling based on workload. AKS will either add or subtract containers as needs fluctuate. Using Kubernetes Event-Driven Autoscaling (KEDA), you can define how these changes occur in response to resource demands. Traffic spikes may also trigger provisioning.

Azure calls this “elastic bursting.” The concept is that workloads should only consume the resources needed to stay afloat. AKS can responsively summon additional compute. Accordingly, uptime sits at 99.9% or 99.95% (with an SLA agreement). Control plane components are split across clusters in specified zones.

Finally, security is a major focus area. Users inherently have privileges, which define accessibility to resources and functionality. Integration with Azure Active Directory means admins can configure identity and access management across their infrastructure. That fine-grained control prevents unauthorized access. It also allows customized control from one cluster to the next.

Azure Private Link also works to limit AKS control plane access to Azure networks. Communication pathways are protected. Interaction between nodes will be safeguarded from prying eyes, thanks to enterprise-grade encryption at rest. Admins may manage keys and disk images.

Pricing & Support

AKS employs a pay-as-you-go pricing model for nodes. You can even choose capacity reservations for one year or three years. Spot resources are also available. It does cost $0.10 per cluster, per hour, when adding an Uptime SLA. The service also provides a cost calculator to determine costs based on your usage. Luckily, there are no charges for cluster management with AKS.

On the support side, Microsoft offers Azure Advisor to optimize both savings and performance. This resource provides guidance based on your needs. Users can submit support requests via their account, and AKS forums provide supplemental help.

Amazon Elastic Kubernetes Service (EKS)

EKS is the new kid on the block, launching shortly after AKS back in 2018. The story here is the same; Amazon EKS aims to bolster scalability, security, and visibility into your Kubernetes deployment.

Compatibility

Amazon EKS supports the following K8s versions:

• 1.18.9
• 1.17.12
• 1.16.15
• 1.15.12

Amazon warns those updating a cluster from 1.16 to 1.17. The update will fail if any AWS Fargate pods are running a kubelet version below 1.16. You must recycle these pods until they’re running v1.16. Subsequent update attempts will then be successful. Additionally, Amazon recommends that you choose the latest EKS Kubernetes version that your clusters may support (unless a specific version is necessary). Support for Kubernetes 1.19 is currently slated for February 2021.

Meanwhile, EKS OS support is somewhat more proprietary:

• Amazon Linux 2 (default)
• Ubuntu Amazon Machine Image (AMI)
• Windows Server 2019

Amazon requires that you use their distributions on EKS. Otherwise, Windows Server support is on par with competing offerings. Elastic Kubernetes Service also mandates that you use Docker for container runtimes—thus more limited when compared to Google and Azure.

Feature Highlights

Orchestration is unique on EKS, thanks in part to eksctl—Amazon’s open-source command line tool. It’s unique to EKS clusters and streamlines the creation of your clusters. Users can install eksctl using three methods:

• Homebrew, for macOS
• Curl, for Linux
• Chocolatey, for Windows

Eksctl is most commonly used to create or delete clusters and node groups. However, it’s also useful for scaling, updating, configuring, listing, installing, and writing config files. All commands submit a CloudFormation query with all required resources. This abstraction is user friendly. You can also drain nodes via this interface, to keep applications running harmoniously.

Node management is also a standout feature. EKS supports both Windows worker nodes and Windows container scheduling. You can manage applications with the same cluster—since EKS permits tandem operation of Windows worker nodes and Linux nodes. Upgrades are user initiated.

Another functionality worth highlighting is the Cluster Autoscaler which automates the adding and removal of nodes in a cluster based on pod behavior. This feature leverages both the Kubernetes Cluster Autoscaler which is a core component of the control panel, as well as the AWS Auto Scaling group (ASG).

Interestingly, EKS supports Arm-based EC2 instances. These are powered by AWS Graviton2 processors. The primary benefit to this arrangement, Amazon claims, is exceptional performance per dollar. Clusters supported by x86 architectures don’t quite measure up. Note that these instances are only available in certain regions, so not all customers may leverage them.

Networking and connecting applications to your clusters are also easy. AWS App Mesh and its controller let you create new meshed services, determine load balancing, and add encryption. App Mesh integrates with AWS Cloud Map—which permits custom resource naming. This not only simplifies management, but it allows web services to locate your updated application components more quickly.

EKS’ control planes are distributed across numerous availability zones. EKS uses three zones at any given time to boost uptime—which sits at 99.95% on average. No SLA is needed to secure this rate.

You may use Fargate for your deployment. Fargate is designed to outsource the provisioning and management of cluster nodes. You simply provision containers based on your sizing requirements and let AWS do the provisioning of nodes in the cluster to accommodate your requirements.. Similarly to standard EKS, Fargate also supports RBAC, IAM, and a virtual private cloud. This feature set brings EKS more in line with options like AKS—which integrates with Active Directory.

Pricing & Support

EC2 instances via EKS cost $0.10 per cluster, per hour. Fargate resources are conversely pay-as-you-go, and the AWS Pricing Calculator will help estimate your overall investment. Want to know how EKS financially compares to your existing deployment? Swing by Amazon’s Economics Resource Center to learn more.

You may contact Amazon for assistance via phone or email. You may also file a formal support ticket for EKS issues, or consult the Knowledge Center for casual problem solving.