How to Deploy a K8s Cluster Using Kubespray

Kubespray is a powerful open source tool for deploying and managing Kubernetes clusters that provides a balance of implementation flexibility and ease of use. The tool works with public cloud, on-premises, bare-metal, and test environment solutions, making it ideal for managing highly available clusters across multiple different platforms.

Ideal uses for Kubespray include:

• Deploying production-grade Kubernetes clusters in On-Prem and Bare-Metal environments that lack sophisticated managed solutions such as GKE/EKS/AKS.
• Deploying production-grade Kubernetes clusters in Cloud environments without losing control over Kubernetes Control Plane Components to a managed cloud provider.

In this article, we’ll take a quick look at Kubespray’s features before diving into an example deployment workflow.

How does Kubespray Work?

Kubespray utilizes a combination of Ansible and Kubeadm to deploy a Kubernetes cluster. It is extremely composable, meaning that you can choose from a wide range of options for network plugins, Linux distributions, and container runtimes. See the below table for a list of options:

Before using Kubespray, we recommend becoming familiar with Ansible constructs like Playbooks and Inventory.

Prerequisites

1. AWS CLI must be installed
2. Terraform must be installed

Deployment Workflow

1. Download & Install Kubespray

1. Create a workspace folder on your local machine:
   

   mkdir ~/Projects/kubespray_workspace
     cd ~/Projects/kubespray_workspace

2. Download the latest stable release of Kubespray:
   

   wget https://github.com/kubernetes-sigs/kubespray/archive/refs/tags/v2.15.1.tar.gz

3. Extract Kubespray
   

   tar -xvzf v2.15.1.tar.gz

4. Navigate into the Kubespray folder and locate the requirements.txt file. This file lists all dependencies.
5. Run the following command to install all dependencies.
   

   sudo pip install -r requirements.txt

2. Provision Infrastructure

As previously mentioned, you can choose your favorite combination of environments and infrastructure provisioning tools. Infrastructure topology should be based on organizational needs (e.g., the number of master or worker nodes, firewalls, subnet CIDR ranges, etc.).

For this article, we’ll use an AWS EC2 VM as the infrastructure. You can reference sample Terraform scripts within your newly-extracted Kubespray folder by navigating to ~/Projects/kubespray_workspace/kubespray-2.15.1/contrib/terraform/aws/.

1. Log in to the AWS Console
2. Download the following AWS Credentials:
   1. AccessKeyID
   2. SecretAccessKey
   3. EC2 Key Pairs
3. Update credentials.tfvars:
   

   cd ~/Projects/kubespray_workspace/kubespray-2.15.1/contrib/terraform/aws/ cp credentials.tfvars.example credentials.tfvars 

4. Update terraform.tfvars based on your infrastructure needs.
5. Run the following Terraform scripts to provision:
   

   terraform init
     terraform plan -out kubesprayplan -var-file=credentials.tfvars
     terraform apply "kubesprayplan"

6. Verify the generated host inventory file:
   

   cat ~/Projects/kubespray_workspace/kubespray-2.15.1/inventory/hosts

7. Verify the SSH bastion host config file:
   

   ~/Projects/kubespray_workspace/kubespray-2.15.1/ssh-bastion.conf

3. Deploy Your Kubernetes Cluster

1. Load the ssh private key obtained from AWS as part of your EC2 key pair.
   

   cp test_aws_instance.pem ~/.ssh/test_aws_instance.pem
     chmod 600 ~/.ssh/test_aws_instance.pem
     eval $(ssh-agent)
     ssh-add -D
     ssh-add ~/.ssh/test_aws_instance.pem 
     

2. Navigate to the extracted Kubespray root folder:
   

   ~/Projects/kubespray_workspace/kubespray-2.15.1

3. Run the following Ansible command to install your kubernetes components on top of your infrastructure:
   

   ansible-playbook -i ./inventory/hosts ./cluster.yml -e ansible_user=ubuntu -b --become-user=root

4. Access Your Kubernetes Cluster

Via Bastion Host

1. Run the Following:
   

   cd ~/Projects/kubespray_workspace/kubespray-2.15.1
     ssh -F ssh-bastion.conf ubuntu@ 
     sudo cat /etc/kubernetes/admin.conf
     mkdir ~/.kube/
     sudo cp -R /etc/kubernetes/admin.conf ~/.kube/config
     sudo chown ubuntu:ubuntu ~/.kube/config
     

2. Verify using the following:
   

   kubectl get nodes
     kubectl get ns

Via Local Machine

1. Run the following:
   

   mkdir ~/.kube/

2. Copy the contents of admin.conf:
   

   /etc/kubernetes/admin.conf

3. Paste in the contents in the following file:
   

   ~/.kube/config

4. Edit the file and replace server: https://127.0.0.1:6443 with the following:
   

   server: https://<aws_elb_dns_name>:6443

5. Verify using the following:
   

   kubectl get nodes
     kubectl get ns

How to Perform 5 Basic Cluster Operations

Remove a Kubernetes Node

ansible-playbook -i ./inventory/hosts ./remove-node.yml -e ansible_user=ubuntu -b --become-user=root --extra-vars "node=ip-10-250-212-108.ap-south-1.compute.internal"

Add The Same Kubernetes Node Back

ansible-playbook -i ./inventory/hosts ./scale.yml -e ansible_user=ubuntu -b --become-user=root

Verify the Kubernetes Version

kubectl version

Upgrade the Kubernetes Version

ansible-playbook -i ./inventory/hosts ./upgrade-cluster.yml -e ansible_user=ubuntu -b --become-user=root -e kube_version=<replace_kubernetes_version_you_want_to_upgrade>

Decommission or Cleanup Infrastructure

cd ~/Projects/kubespray_workspace/kubespray-2.15.1/contrib/terraform/aws/
  terraform destroy -var-file=credentials.tfvars

Upgrading Kubespray

Kubespray does not support or recommend skipping any release versions when upgrading. If you are using, for example, version 1.0.0 and want to upgrade to 5.0.0, you must upgrade to 2.0.0, 3.0.0, and so on for every official Kubespray release.

Conclusion

While there are many Kubernetes tools available today, few offer the same level of platform flexibility provided by Kubespray. Kubespray is a great solution for organizations that are already familiar with or are actively using Ansible for their existing provisioning and orchestration. In addition, Kubespray’s composability means that your team gets to choose the tech (applications, Kubernetes runtime, Linux distribution, network plugins, etc.) that you already love working with. If your organization anticipates needing a multi-platform strategy (across cloud, bare-metal, on-prem, and others) and has already adopted Ansible, then Kubespray is an ideal choice for deploying your Kubernetes clusters.