ResourceQuota is an object in Kubernetes that enables administrators to restrict cluster tenants’ resource usage per namespace. Namespaces create virtual clusters within a physical Kubernetes cluster to help users avoid resource naming conflicts and manage capacity, among others.
In this article, we explore Kubernetes resource quota configuration requirements to govern resource consumption by namespace. We have included demo codes illustrating how to use .yaml files with kubectl to configure constraints on hardware usage.
A Kubernetes cluster has a limited amount of available hardware resources. Hardware resources are measured based on worker nodes with a specific number of CPU cores or RAM allocation.
In a shared Kubernetes environment, it is important to pre-define the allocation of resources for each tenant to avoid unintended resource contention and depletion.
By utilizing this method, each tenant in a shared Kubernetes cluster can be configured with specific settings for maximum allowed resource consumption.
To put resource limits on individual containers, use the Kubernetes resource requests and limits associated with pods within each namespace.
A cluster can contain multiple namespaces intended for administrative separation. Each namespace can in turn contain multiple pods of containers. The namespace and the pod each have their own resource configuration files. The namespace resource quota governs the maximum use of computing resources by the namespace, while the pod request and limits govern the use of computing resources by the containers within each pod.
A Kubernetes resource request establishes the amount of computing resources a pod is reserving for itself at creation time. This configuration is important because it helps the Kubernetes scheduler make smart decisions by placing an appropriate number of pods on each node. The resource limit is the upper bound on the CPU or RAM usage a pod can possibly use.
A pod quota can be defined based on resource request and limit values in a YAML file referenced by the Kubernetes JSON API.
Requests for the creation, deletion, and update of system resources go through the Kubernetes API server. There are different admission controllers that can view and filter the requests. The quota operates until the resource limit is reached or violated.
Once the resource quota object is defined on a namespace by the Kubernetes cluster administrator, the Kubernetes quota admission controller watches for the new objects created in that namespace. Then it will keep monitoring and tracking resource usage.
If a user or process tries to acquire more of that resource, the Kubernetes admission controller will throw an error or exception and will not allow that operation.
The following points summarize the main steps to understanding how Kubernetes resource quota configuration works in practice:
ResourceQuota object support is enabled by default on most Kubernetes distributions. It can be enabled manually by setting the API server. For example, the command
--enable-admission-plugins= flag has the ResourceQuota object as its target argument.
A resource quota is enforced in a particular namespace when there is a ResourceQuota object in that namespace. This file can be utilized in different ways in configuration.
In this section, we go through a demo example of how to create and define the CPU resource quota on a namespace with requests and limits.
We will then try to exceed the quota through web traffic spikes to show that once the defined object limit is reached, no more resources using that quota can be created.
First, check to see if you have an active Kubernetes cluster up and running. We have some worker nodes available for the code. At least one worker node is required to perform this demo.
Next, create a namespace to test Kubernetes resource quotas, quota-demo:
Then, define and create the CPU quota on a namespace:
You can verify the ResourceQuota object has been created. Note: the “used” column, as initially no quota is used in the configuration and the namespace is populated as empty.
Thereafter, you can create a test pod with requests and limits defined as shown below:
See the updated settings on the namespace and note the data displayed in the “used” column. You will now notice a difference, with the new pod just created having used some of the quota:
To continue the process, create another pod and observe the remaining quota values:
As noted above, the used column is now updated again. The new pod is now listed as having consumed more of the available quota limits for the total allocated CPU resources.
If we try to create a pod requesting more resources than what’s available in quota, we will now receive an error message that states we don’t have enough quota left to create the new pod:
Finally, do the clean-up of the installation and configuration files by deleting the namespace and resources contained in it. You can use the command below with your configuration variables: