Requests and Limits: The Kubernetes Settings Everyone Gets Wrong

Every Kubernetes container carries two pairs of numbers — a CPU request and limit, a memory request and limit — and most of them were set by copying the last service’s values. That is a shame, because requests and limits are among the highest-leverage settings you have, and getting them wrong quietly wastes money, causes mysterious latency, and triggers 3am OOM kills. The rules are not complicated, but they are counterintuitive, and CPU and memory follow opposite logic.

Requests are for scheduling, limits are for enforcement

Start with what the two words actually mean, because the names are unhelpful. A request is a reservation: it tells the scheduler how much of a node to set aside for this container, and the scheduler will not place a pod on a node that cannot satisfy the sum of its requests. A limit is a ceiling: it tells the kubelet to stop the container from using more than this, once it is running.

So requests decide where a pod runs, and limits decide what happens when it gets greedy. They are entirely different mechanisms, and the gap between a container’s request and its limit is where most of the interesting behaviour lives.

CPU and memory are not the same, and treating them the same is the core mistake

Here is the fact that reframes everything: CPU is compressible and memory is not. When a container hits its CPU limit, the kernel throttles it — slows it down, makes it wait. When a container hits its memory limit, the kernel kills it. One is a speed bump; the other is death. Because the consequences are so different, the two resources deserve opposite strategies.

Memory: always set a limit, set request equal to it

Memory is the straightforward one. Always set a memory limit, because a container without one can consume the whole node and take down every other pod on it — one leaking service becoming an outage for its innocent neighbours. That is not a risk worth running to save five minutes.

The refinement most teams miss: for memory, set the request equal to the limit. This places the pod in the Guaranteed quality-of-service class, which means it is the last thing the kernel kills when the node is under memory pressure. A pod whose memory request is lower than its limit is Burstable, and it is a candidate for eviction the moment the node gets tight — even though it is behaving perfectly. Request-equals-limit for memory trades a little scheduling flexibility for a lot of stability, and for anything that matters it is the right trade.

Size the number from reality, not a guess. Watch the container’s working set under real load for a week, take the peak, and add roughly 25% headroom for spikes and garbage-collection slack. For a JVM specifically, remember the runtime sizes its heap against the limit, so a too-tight limit gets you killed by the kernel while the JVM believes it has room — the memory-diagnosis reasoning in our JVM profiling guide applies directly.

CPU: set a request, and think hard before setting a limit

CPU is where the advice gets contrarian, and where copied config does the most damage. Set a CPU request so the scheduler reserves a fair share and your pod is not starved on a busy node. But a CPU limit is often actively harmful, and this surprises people.

The mechanism is the kernel’s CFS quota. A CPU limit is enforced as a slice of time per 100ms period; exceed it and your process is stopped until the next period begins. For a latency-sensitive service that occasionally needs a burst — handling a request spike, running a GC pause — that throttling shows up as tail-latency spikes with no error and no obvious cause. The p99 gets worse and nothing in your logs explains it.

# Throttling is invisible to kubectl — it lives in cgroup stats
kubectl exec <pod> -- cat /sys/fs/cgroup/cpu.stat | grep throttled
# nr_throttled and throttled_usec climbing = the CPU limit is hurting you

Multi-threaded runtimes suffer most, because a JVM or Go process sees all the node’s cores, sizes its thread pools to match, and then burns its entire quota in a fraction of the period. The common, well-supported position is: set CPU requests to guarantee a share, and leave CPU limits off for latency-sensitive workloads, letting them burst into spare capacity. Keep CPU limits for batch or genuinely untrusted workloads where predictability matters more than speed. This is the exact opposite of the memory rule, and that asymmetry is the whole lesson.

Right-sizing without guessing

The numbers should come from measurement, and the failure mode is over-requesting “to be safe.” A pod that requests four cores and uses half of one does not make itself safer — it strands three and a half cores that no other pod can schedule onto, so a third of your cluster sits reserved and idle. Multiply that across a fleet and you are paying for double the nodes you need.

Watch actual usage with the metrics server or your monitoring stack, and set requests near the real median-to-p90, not the theoretical maximum. A Vertical Pod Autoscaler can recommend values from observed usage, which is a good way to escape the copy-the-last-service habit. And when a pod will not schedule at all, the reason is almost always requests that do not fit — the Insufficient cpu and Insufficient memory messages in our pod troubleshooting guide are the direct consequence of requests set too high.

Namespace defaults so nobody ships a container with no limits

Everything above assumes each container sets sensible values, and in a real team some will not — a service ships with no memory limit, or with a copied CPU limit that throttles it, and nobody notices until production. Two cluster-level objects turn the good advice into an enforced default rather than a hopeful convention.

A LimitRange in a namespace sets default requests and limits for any container that omits them, and can enforce minimums and maximums. With one in place, a pod deployed with no memory limit does not get an unbounded one — it inherits the namespace default, so the worst case is capped even when a team forgets. It also lets you forbid the pathological cases outright, such as a limit far larger than the request, which is the configuration most likely to cause surprise evictions.

apiVersion: v1
kind: LimitRange
metadata: {name: sane-defaults}
spec:
  limits:
    - type: Container
      default: {memory: 512Mi}          # limit if none specified
      defaultRequest: {memory: 256Mi, cpu: 100m}
      max: {memory: 4Gi}                # nothing may exceed this

A ResourceQuota works at the other end, capping the total requests and limits a whole namespace may consume, so a single team cannot accidentally reserve the entire cluster. Together they turn resource hygiene from something every engineer must remember into something the platform guarantees — a default that holds even when the individual container config is wrong. This is the same philosophy as the namespace-level guards in our multi-tenant isolation guide: the safe behaviour should be the one you get by default, not the one you have to remember to configure. On a shared cluster these two objects prevent more incidents than any amount of per-container diligence, precisely because they do not depend on diligence.

The rules, condensed

Always set a memory limit, and set the memory request equal to it for anything that matters. Always set a CPU request, and be very reluctant to set a CPU limit on latency-sensitive services. Size every number from observed usage plus modest headroom, never from a copied template or a fearful guess. And when latency mysteriously worsens with no errors, check cpu.stat for throttling before you blame the application — that one habit will save you a debugging session or two a year.

None of this is exotic, but almost every cluster gets it wrong in the same way: uniform copied values, memory requests below limits, and CPU limits throttling the very services that most need to burst. Fixing it is mostly deletion — removing CPU limits and correcting memory QoS — which makes it the rare optimisation that simplifies your config while it speeds things up.