Kubernetes Node Affinity: Teaching Your Pods Where to Live
Kubernetes scheduling often looks effortless in early learning. You deploy a workload, pods appear, and everything seems under control until real production traffic arrives and suddenly the scheduler’s “autopilot” decisions start causing surprises.
In real clusters, nodes are far from identical. Some are lightweight, some are memory-rich, some include GPUs, some run on spot pricing, some exist in specific regions, and some are reserved for sensitive workloads. When placement is left entirely to default scheduling, sooner or later you’ll face problems that appear to be application issues, but are actually workload placement problems.
Node affinity is how you replace guesswork with intention. It gives Kubernetes clear guidance on which nodes are suitable (or preferable) for each pod ensuring workloads run where they make operational sense.
Let’s explore how it works clearly and practically.
What does Node Affinity mean?
Node affinity is a scheduling rule set attached to a pod that influences which nodes the Kubernetes scheduler is allowed to consider, using node labels as selection criteria.
In simple terms, you’re telling Kubernetes:
“Only schedule this pod on nodes matching these labels.”
or
“Try these nodes first, but fall back if necessary.”
This becomes essential when your cluster contains:
- GPU nodes for machine learning
- High-memory nodes for databases
- Spot instances for interruptible tasks
- SSD-backed nodes for storage-intensive services
Without affinity rules, Kubernetes might schedule a GPU-dependent workload onto a standard node or place a critical database on unstable spot capacity both leading to avoidable failures.
The Two Node Affinity Modes (When to Use Which)
1) requiredDuringSchedulingIgnoredDuringExecution (Hard Requirement)
This mode enforces strict placement conditions.
If no matching node exists, the pod simply remains unscheduled.
Use it when placement cannot be compromised:
- Regulatory or data-residency constraints
- Mandatory hardware requirements (GPU / SSD / high-memory)
- Workloads that cannot function on unsuitable nodes
Example: only run on SSD nodes
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- ssd
Meaning: This workload is allowed only on SSD-backed nodes. If none exist, it waits.
2) preferredDuringSchedulingIgnoredDuringExecution (Soft Preference)
This mode expresses placement preference rather than enforcement.
Kubernetes attempts to satisfy the preference, but will still schedule the pod elsewhere if needed.
Use it when optimization is desired without blocking deployments:
- Favor specific zones while tolerating others
- Prefer cost-efficient spot nodes with safe fallback
- Encourage better performance without risking Pending pods
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- ssd
Higher weight = stronger preference when multiple preferences compete.
The 6 Node Affinity Operators You’ll Actually Use
Affinity rules rely on matchExpressions, which use operators to define how node labels are evaluated.
1) In - match specific values
Schedules pods on nodes whose label value matches any listed option.
- key: instance-type
operator: In
values: [t3.large, t3.xlarge, m5.large]
2) NotIn - exclude values
Prevents scheduling on nodes with specific label values.
- key: node-lifecycle
operator: NotIn
values: [spot]
3) Exists - label key must be present
Node must contain the label key; the value itself is irrelevant.
- key: production-ready
operator: Exists
4) DoesNotExist - label key must be absent
Ensures pods avoid nodes marked with certain labels.
- key: deprecated
operator: DoesNotExist
5) Gt - numeric “greater than”
Label value must be numerically greater than a defined threshold.
- key: memory-gb
operator: Gt
values: ["32"]
6) Lt - numeric “less than”
Label value must be numerically less than a defined threshold.
- key: cpu-cores
operator: Lt
values: ["4"]
Together, these operators allow expressive and precise node selection logic.
Real Production Scenarios Where Node Affinity Saves You
Scenario 1: ML Workloads Requiring GPUs
Ensure training or inference pods run only on GPU-enabled nodes.
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: accelerator
operator: In
values: [nvidia-tesla-t4, nvidia-tesla-v100]
Scenario 2: Region-Specific Compliance
Keep workloads inside approved regions for data governance requirements.
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: topology.kubernetes.io/region
operator: In
values: [eu-west-1, eu-central-1]
Scenario 3: Cost optimization using spot capacity
Prefer spot instances for savings, while allowing fallback to standard nodes.
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: node-lifecycle
operator: In
values: [spot]
Scenario 4: Databases with high I/O needs
Prioritize SSD-backed nodes to maintain consistent database performance.
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 90
preference:
matchExpressions:
- key: disktype
operator: In
values: [ssd]
Each scenario selects required or preferred affinity based on strictness.
Combining Rules Correctly: AND vs OR (The Part That Trips People)
AND logic: Multiple Expressions in One Term
All listed conditions must be satisfied for a node to qualify.
nodeSelectorTerms:
- matchExpressions:
- key: zone
operator: In
values: [us-west-1a]
- key: disktype
operator: In
values: [ssd]
- key: instance-size
operator: In
values: [large, xlarge]
OR logic: multiple nodeSelectorTerms
If any term matches, the node is acceptable.
nodeSelectorTerms:
- matchExpressions:
- key: zone
operator: In
values: [us-west-1a]
- key: disktype
operator: In
values: [ssd]
- matchExpressions:
- key: zone
operator: In
values: [us-east-1b]
- key: disktype
operator: In
values: [nvme]
This structure enables advanced placement strategies without complex scripting.
Here are some Common Mistakes (And Practical Fixes)
Mistake 1: Making Every Rule “Required”
Excessive strictness can leave pods unscheduled during capacity shifts.
Fix: Reserve required affinity for true non-negotiables.
Use preferred affinity for optimizations.
Mistake 2: Forgetting to Label Nodes
Affinity rules cannot match unlabeled nodes.
Fix: Label nodes first, then define affinity rules.
kubectl label nodes node-1 disktype=ssd
kubectl label nodes node-1 zone=us-west-1a
kubectl label nodes node-1 instance-type=m5.xlarge
Mistake 3: Overcomplicated Logic
Highly nested conditions become difficult to maintain.
Fix: Keep rules understandable at a glance.
Mistake 4: Ignoring Weight Values
Equal weights eliminate meaningful preference order.
Fix: Assign higher weights to strong priorities.
How Node Affinity Works With Other Scheduling Tools
Node Affinity + Taints/Tolerations
- Taints prevent general workloads from landing on specific nodes.
- Node affinity attracts intended workloads to those nodes.
Used together, they enforce clean workload separation.
Node affinity + pod (anti)affinity
- Node affinity selects which nodes pods run on.
- Pod affinity controls which pods run near each other.
- Pod anti-affinity keeps pods strategically separated.
Together, they shape deliberate cluster topology.
Ask Yourself
Q1) nodeSelector vs node affinity?
nodeSelector provides basic exact label matching.
Node affinity enables richer operators and flexible scheduling logic.
Q2) What if node labels change after pods start running?
Nothing changes automatically.
Affinity rules are evaluated during scheduling, not after runtime begins.
Q3) When should required vs preferred be used?
Required → When workloads must run only on specific nodes.
Preferred → When better placement is desired but fallback is acceptable.
Conclusion
Node affinity is foundational for reliable production Kubernetes operations.
It brings predictability to scheduling, stability to workloads, and clarity to infrastructure usage.
- Define labels carefully.
- Apply strict rules only where necessary.
- Use preferences where flexibility is acceptable.
- Validate behavior before production rollout.
When you control placement, Kubernetes stops being reactive and becomes intentional.
“Good clusters don’t guess. They schedule with intent.”