Kubernetes Pod Resizing: Scale CPU and Memory Without Restarts
Kubernetes pod resizing solves one of the most frustrating operational problems in container orchestration: changing a pod’s resource allocation traditionally requires killing it and starting a new one. For stateful workloads, websocket connections, and long-running batch jobs, this forced restart means downtime, lost connections, and interrupted work. In-place pod resizing changes that entirely by letting the kubelet adjust resources on a running container.
The Problem This Solves — Why It Matters
Consider a real scenario: your API server normally uses 500m CPU and 512Mi memory. During a marketing campaign, traffic spikes 5x and the pods need 2 CPU cores and 2Gi memory. With traditional Kubernetes, the only option is to update the Deployment spec and trigger a rolling restart — killing active connections, dropping in-flight requests, and causing a brief service disruption right when traffic is highest.
With in-place resizing, the kubelet adjusts the container’s cgroup limits while it continues running. The process inside the container never knows anything changed. Moreover, active TCP connections, in-memory caches, and JVM warmup state are all preserved. Consequently, resource scaling becomes as seamless as adjusting a thermostat instead of rebuilding the furnace.
This is especially critical for:
- Stateful databases — PostgreSQL, Redis, MongoDB pods that hold connection state
- WebSocket servers — Chat applications, real-time dashboards with thousands of active connections
- ML inference — GPU-accelerated pods that take minutes to load models into memory
- Batch processing — Jobs that have been running for hours and can’t be restarted
How It Works Under the Hood
When you patch a pod’s resource requests/limits, the kubelet communicates with the container runtime (containerd or CRI-O) to update the cgroup v2 limits. For CPU, this is nearly instantaneous — the kernel simply adjusts the CPU bandwidth allocation. For memory, it’s more nuanced because the kernel can’t always reclaim memory that’s already been allocated to the process. Therefore, the behavior differs sharply between the two resources, and the API surfaces that difference through resize policies.
# Deployment with resize policies configured
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
namespace: production
spec:
replicas: 3
template:
spec:
containers:
- name: api
image: api-server:v2.4.1
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "2000m"
memory: "2Gi"
resizePolicy:
- resourceName: cpu
restartPolicy: NotRequired # CPU changes: no restart needed
- resourceName: memory
restartPolicy: RestartContainer # Memory increases MAY need restart
---
# To resize a running pod (kubectl 1.33+):
# kubectl patch pod api-server-7d9f8b6c4-x2k9p --subresource resize -p # '{"spec":{"containers":[{"name":"api","resources":{"requests":{"cpu":"1000m","memory":"1Gi"},"limits":{"cpu":"4000m","memory":"4Gi"}}}]}}'
The resizePolicy field is critical. For CPU, NotRequired works on all major runtimes because CPU bandwidth is a kernel-level limit that applies immediately. For memory, it’s more complex: increasing the limit is always safe (you’re just allowing the process to use more), but the container runtime may need to restart the container to apply certain memory configurations. Therefore, RestartContainer is the conservative default for memory until you have validated your runtime’s behavior.
Feature Gate and Version Status
Before relying on this in any environment, confirm where the feature stands in your cluster. In-place resize shipped as an alpha feature behind the InPlacePodVerticalScaling feature gate and progressed to beta in Kubernetes 1.33, where the gate is enabled by default. On older clusters you must enable the gate explicitly on the API server and kubelet, and you should expect API shape changes across versions — the dedicated resize subresource, for instance, is what you target rather than a plain pod patch.
# Confirm the feature gate state on a node's kubelet
kubectl get --raw "/api/v1/nodes/$NODE/proxy/configz" | \
python3 -c "import sys,json; \
print(json.load(sys.stdin)['kubeletconfig'].get('featureGates', {}))"
# On self-managed clusters running an older version, enable explicitly:
# kube-apiserver --feature-gates=InPlacePodVerticalScaling=true
# kubelet --feature-gates=InPlacePodVerticalScaling=true
Because the precise behavior and defaults shift between releases, always cross-check the changelog for your exact version rather than assuming parity with another cluster. A common pattern is to pin the Kubernetes version in staging to match production so resize behavior is identical across both.
Vertical Pod Autoscaler (VPA) Integration
The real power of in-place resizing comes when combined with VPA. Previously, VPA had a frustrating chicken-and-egg problem: it could calculate the right resource values but could only apply them by evicting the pod and letting it restart with new values. This caused disruptions and was often disabled in production for exactly that reason.
With the newer updateMode: InPlaceOrRecreate (available in recent VPA releases), the autoscaler applies recommendations in place where possible and falls back to recreation only when necessary. Additionally, you can configure bounds to prevent VPA from over-allocating:
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: api-server-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: api-server
updatePolicy:
updateMode: "InPlaceOrRecreate" # Resize in place, recreate if needed
minReplicas: 2 # Keep at least 2 pods during updates
resourcePolicy:
containerPolicies:
- containerName: api
minAllowed:
cpu: "250m"
memory: "256Mi"
maxAllowed:
cpu: "4000m"
memory: "8Gi"
controlledResources: ["cpu", "memory"]
controlledValues: RequestsAndLimits
In practice, VPA plus in-place resizing means your pods automatically right-size throughout the day. A service that needs 2 cores during peak hours and 500m during off-hours adjusts without any human intervention and, in the best case, without pod disruption.
Kubernetes Pod Resizing: Monitoring and Troubleshooting
After submitting a resize, check the pod’s status to see whether it was applied. The API exposes resize state through pod conditions and status fields, so observability is just a matter of querying the right path:
# Check resize status condition (PodResizePending / PodResizeInProgress)
kubectl get pod api-server-7d9f8b6c4-x2k9p \
-o jsonpath='{.status.conditions[?(@.type=="PodResizePending")].reason}'
# Check actual allocated resources (what the container currently has)
kubectl get pod api-server-7d9f8b6c4-x2k9p \
-o jsonpath='{.status.containerStatuses[0].allocatedResources}'
# Monitor resize events on the pod
kubectl get events --field-selector \
involvedObject.name=api-server-7d9f8b6c4-x2k9p
A Deferred/Pending reason means the node doesn’t have enough resources right now — the resize will be applied when capacity frees up. Infeasible means it can never be satisfied on this node, for example requesting more CPU than the node physically has. However, in most production clusters with healthy resource margins, resizes complete within seconds.
A Realistic Walkthrough: Resizing a Live JVM Service
To make this concrete, walk through scaling a Java service whose heap pressure is climbing under load. First, confirm current usage so you do not shrink memory below what the process already holds. Next, patch only the resource you intend to change, and finally verify that the container reports the new allocation without a restart count increment.
# 1. Inspect current usage before changing anything
kubectl top pod api-server-7d9f8b6c4-x2k9p
# 2. Bump CPU in place (no restart for CPU under NotRequired)
kubectl patch pod api-server-7d9f8b6c4-x2k9p --subresource resize \
--type='strategic' -p \
'{"spec":{"containers":[{"name":"api","resources":{"requests":{"cpu":"1500m"},"limits":{"cpu":"3000m"}}}]}}'
# 3. Confirm restartCount did NOT increase (proves no restart happened)
kubectl get pod api-server-7d9f8b6c4-x2k9p \
-o jsonpath='{.status.containerStatuses[0].restartCount}'
If restartCount is unchanged after a CPU resize, the operation was truly in place and every connection survived. For a JVM specifically, the newly granted cores become available to the scheduler immediately, and modern JVMs that read CPU quota dynamically will widen their thread pools accordingly — one of the clearest wins of in-place CPU scaling.
Production Best Practices
1. Always set resize policies explicitly. Don’t rely on defaults — be clear about which resources can resize without restart. This prevents surprises in production.
2. Set reasonable maxAllowed limits. A pod that auto-scales to 32 cores on a 64-core node can starve other workloads. Use VPA resource policies to cap growth.
3. Combine with HPA wisely. Use HPA for horizontal scaling (add more pods) and VPA for vertical scaling (make existing pods bigger). For most workloads, HPA handles traffic spikes while VPA optimizes baseline resource allocation. Furthermore, configure HPA to scale on custom metrics such as requests per second rather than CPU, since VPA is also changing CPU and the two controllers would otherwise fight.
4. Test resize behavior in staging first. Some applications don’t handle cgroup limit changes gracefully. Java’s JVM respects dynamic CPU count changes, but it won’t return allocated heap memory when memory limits decrease. Go applications generally handle it well, and Node.js is usually fine. Specifically, validate your runtime’s behavior before enabling in production.
5. Monitor for OOMKill after memory resizes. If you decrease memory limits while the process is already using more than the new limit, the kernel will OOMKill the container. Always check current usage with kubectl top before decreasing limits.
When NOT to Use In-Place Resizing — Trade-offs and Limits
In-place resizing is powerful, but it is not the right answer for every scaling problem, and reaching for it reflexively can hide deeper issues. If your workload is stateless and horizontally scalable, adding replicas with HPA is usually simpler, safer, and gives you redundancy as a bonus — a single bigger pod is still a single point of failure. Vertical resizing shines specifically when replicas are expensive or impossible to add, as with stateful databases or GPU pods, not as a default scaling strategy.
Be honest about the constraints as well. The feature works only with cgroup v2, so cgroup v1 nodes must be upgraded first. You cannot resize ephemeral storage — only CPU and memory — and init containers cannot be resized. Memory increases are smooth, but memory decreases are genuinely risky because most runtimes never hand reclaimed heap back to the kernel, so a shrink can trigger an OOMKill rather than a graceful contraction. Furthermore, managed offerings (EKS, GKE, AKS) expose the feature gates on their own timelines, so a capability available in upstream Kubernetes may lag on your provider. As a rule, prefer horizontal scaling for stateless services, reserve in-place vertical resizing for connection-sensitive or stateful workloads, and never decrease memory without first confirming live usage. Used within those boundaries, it removes real operational pain; used as a blunt instrument, it simply relocates the failure.
Current Limitations (As of Kubernetes 1.33+)
In-place resizing has matured significantly but still has some limitations to be aware of:
- Only works with cgroup v2 — if your nodes use cgroup v1, you need to upgrade first
- Cannot resize ephemeral storage — only CPU and memory
- Init containers cannot be resized
- Some managed Kubernetes services (EKS, GKE, AKS) may have different feature gate availability — check your provider’s documentation
Related Reading:
- Kubernetes Autoscaling with KEDA — Event-Driven Scaling
- Building Custom Kubernetes Operators
- Platform Engineering: Developer Portal Guide
Official Resources:
In conclusion, Kubernetes pod resizing transforms resource management from a disruptive, restart-dependent process into a seamless runtime operation. For any team running stateful or connection-sensitive workloads, this capability alone can justify upgrading to a recent Kubernetes version — provided you respect its limits around memory decreases, cgroup v2, and managed-platform availability.