Every few months someone asks me this question. Usually right after they've just picked one without fully understanding the other.
"Should we use Karpenter or Fargate?"
The honest answer is: it depends on what you're trying to not think about.
Because both of them are trying to solve the same problem — getting your pods running without manual node management — but they solve it from completely different angles, with completely different tradeoffs. Getting this decision wrong means either paying too much, or fighting your infrastructure for years.
π€ The Situation (Real Talk)
I've seen teams choose Fargate because it sounded "serverless" and they didn't want to deal with EC2. Fine. Then six months later they're frustrated because their batch jobs cost three times as much, their DaemonSets don't work, and they can't run privileged containers for their log shipper.
I've also seen teams go all-in on Karpenter when Fargate was exactly what they needed — a simple internal tool that didn't justify owning a node fleet.
Both are wrong. Neither is always right. Let's actually break it down.
π§ What Each One Actually Is
Think of it this way. You're running a restaurant.
Karpenter is like building your own kitchen. You choose the equipment, you hire the cooks, you control everything. It's more work to set up, but you can make any dish you want, at any volume, with exactly the cost you'd expect if you're paying attention.
Fargate is like renting a ghost kitchen per dish. You send in an order, they cook it, you pay per plate. No kitchen to manage. But you're paying a premium per dish, you can't modify the kitchen equipment, and some recipes just don't work there.
⚙️ Side-by-Side Comparison
| Feature | Karpenter | Fargate |
|---|---|---|
| Node management | You (automated) | AWS (fully managed) |
| DaemonSets | ✅ Full support | ❌ Not supported |
| Privileged containers | ✅ Yes | ❌ No |
| GPU workloads | ✅ Yes | ❌ No |
| Cost model | EC2 pricing (Spot available) | Per-pod vCPU + memory pricing |
| Spot savings | ✅ Up to 90% | ❌ No Spot option |
| Pod isolation | Shared node | Dedicated micro-VM per pod |
| Startup time | ~30-90s new node | ~45-90s cold start |
| Custom AMIs | ✅ Full control | ❌ No |
| Security posture | Depends on config | Strong pod isolation by default |
π When Karpenter Wins
Use Karpenter when your workloads are cost-sensitive, complex, or need control.
- High-throughput APIs where you need Spot to keep costs sane
- ML training jobs that need GPU nodes
- Batch processing at scale where per-pod pricing would bankrupt you
- Any workload that needs DaemonSets — log shippers, security agents, monitoring exporters
- Teams who want to actually understand their infrastructure
# Example: GPU NodePool for ML workloads - only Karpenter can do this
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: gpu-pool
spec:
template:
spec:
requirements:
- key: node.kubernetes.io/instance-type
operator: In
values: ["g4dn.xlarge", "g4dn.2xlarge", "g5.xlarge"]
- key: karpenter.sh/capacity-type
operator: In
values: ["on-demand"]
nodeClassRef:
group: karpenter.k8s.aws
kind: EC2NodeClass
name: gpu-class
π When Fargate Wins
Use Fargate when simplicity and isolation matter more than cost optimization.
- Internal tools and low-traffic apps where you don't want to manage nodes at all
- Compliance-heavy workloads that need strong pod-level isolation (each pod gets its own VM boundary)
- Teams with no Kubernetes operational experience yet
- Short-lived CI jobs where you want zero node management overhead
- Multi-tenant environments where noisy-neighbour isolation is critical
# Fargate profile example — this is how simple it is
apiVersion: eks.amazonaws.com/v1alpha1
kind: FargateProfile
metadata:
name: internal-apps
spec:
clusterName: my-cluster
podExecutionRoleArn: arn:aws:iam::123456789:role/FargateExecutionRole
selectors:
- namespace: internal-tools
- namespace: compliance-apps
⚠️ Common Mistakes
- Using Fargate for stateful workloads — Fargate doesn't support EBS volumes directly. Stateful apps on Fargate is a bad time.
- Using Karpenter for tiny, simple apps because "it's more powerful" — Sometimes a Honda Civic is the right vehicle. A Formula 1 car is technically better, but you don't need to know how to change tyres at 200mph just to get to the shops.
- Mixing Karpenter and Fargate without clear namespace boundaries — Profile selectors and NodePool affinities must be explicit or pods land in unexpected places.
- Expecting Fargate to be cheaper at scale — Fargate pricing is per-pod. At volume, EC2 via Karpenter with Spot is significantly cheaper. Do the math before committing.
π₯ CloudChef Pro Tip
You don't have to choose just one.
π Run Fargate for your low-traffic compliance namespace. Run Karpenter for everything else. EKS supports both simultaneously. Use them for what they're actually good at.
Do the cost math first.
π Take your average pod resource requests. Multiply by Fargate pricing. Then compare against what Spot instances via Karpenter would cost for the same workload. In most cases, at moderate scale, Karpenter with Spot wins by 40-60%.
π Continue Your CloudChef Journey
π References
π Final Thoughts
Karpenter vs Fargate isn't really a competition. It's a question of what your team values and what your workloads need.
π If you need control, Spot savings, DaemonSets, or GPU — pick Karpenter. If you need simplicity, strong pod isolation, and don't care about cost optimization — Fargate is a legitimate choice. And if your answer is "both apply to different parts of our cluster" — congratulations, you're thinking about this correctly.
No comments:
Post a Comment