When several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources.
Resource quotas are a tool for administrators to address this concern. Resource quotas work like this:
403 FORBIDDEN
.403 FORBIDDEN
. Hint: Use the LimitRange admission controller to force default
values of limits (then resource requests would be equal to limits by default, see
admission controller) before the quota is checked to avoid this problem.Examples of policies that could be created using namespaces and quotas are:
In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces, there may be contention for resources. This is handled on a first-come-first-served basis.
Neither contention nor changes to quota will affect already-running pods.
Resource Quota support is enabled by default for many Kubernetes distributions. It is
enabled when the apiserver --admission-control=
flag has ResourceQuota
as
one of its arguments.
Resource Quota is enforced in a particular namespace when there is a
ResourceQuota
object in that namespace. There should be at most one
ResourceQuota
object in a namespace.
The total sum of compute resources requested by pods in a namespace can be limited. The following compute resource types are supported:
ResourceName | Description |
---|---|
cpu | Total cpu requests of containers |
memory | Total memory requests of containers |
For example, cpu
quota sums up the resources.requests.cpu
fields of every
container of every pod in the namespace, and enforces a maximum on that sum.
The number of objects of a given type can be restricted. The following types are supported:
ResourceName | Description |
---|---|
pods | Total number of pods |
services | Total number of services |
replicationcontrollers | Total number of replication controllers |
resourcequotas | Total number of resource quotas |
secrets | Total number of secrets |
persistentvolumeclaims | Total number of persistent volume claims |
For example, pods
quota counts and enforces a maximum on the number of pods
created in a single namespace.
You might want to set a pods quota on a namespace to avoid the case where a user creates many small pods and exhausts the cluster’s supply of Pod IPs.
Kubectl supports creating, updating, and viewing quotas:
$ kubectl namespace myspace
$ cat <<EOF > quota.json
{
"apiVersion": "v1",
"kind": "ResourceQuota",
"metadata": {
"name": "quota"
},
"spec": {
"hard": {
"memory": "1Gi",
"cpu": "20",
"pods": "10",
"services": "5",
"replicationcontrollers":"20",
"resourcequotas":"1"
}
}
}
EOF
$ kubectl create -f ./quota.json
$ kubectl get quota
NAME
quota
$ kubectl describe quota quota
Name: quota
Resource Used Hard
-------- ---- ----
cpu 0m 20
memory 0 1Gi
pods 5 10
replicationcontrollers 5 20
resourcequotas 1 1
services 3 5
Resource Quota objects are independent of the Cluster Capacity. They are expressed in absolute units. So, if you add nodes to your cluster, this does not automatically give each namespace the ability to consume more resources.
Sometimes more complex policies may be desired, such as:
Such policies could be implemented using ResourceQuota as a building-block, by writing a ‘controller’ which watches the quota usage and adjusts the quota hard limits of each namespace according to other signals.
Note that resource quota divides up aggregate cluster resources, but it creates no restrictions around nodes: pods from several namespaces may run on the same node.
See a detailed example for how to use resource quota.
See ResourceQuota design doc for more information.