Karpenter

Karpenter (Nov 2021) Kubernetes cluster autoscaling project

• https://karpenter.sh/
• https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html#karpenter

controller.

karpenter.sh

• Releases: https://github.com/aws/karpenter/releases

helm repo add karpenter https://charts.karpenter.sh/
helm repo update

helm upgrade --install --namespace karpenter --create-namespace karpenter karpenter/karpenter \
  --version v0.6.3 \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
  --set clusterName=${CLUSTER_NAME} \
  --set clusterEndpoint=${CLUSTER_ENDPOINT} \
  --set aws.defaultInstanceProfile=KarpenterNodeInstanceProfile-${CLUSTER_NAME} \
  --wait # for the defaulting webhook to install before creating a Provisioner

• kubectl get provisioner
• kubectl describe provisioner
• kind: Provisioner

Logs

• controller.provisioning Relaxing soft constrains ...
• controller.node Triggering termination for expired node after 168h0m0s .../...

kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

kubectl patch configmap config-logging -n karpenter --patch '{"data":{"loglevel.controller":"debug"}}'

stern -n karpenter -l app.kubernetes.io/name=karpenter

News

• Aug 2022 v0.15.0 Workload Consolidation for Karpenter https://aws.amazon.com/about-aws/whats-new/2022/08/workload-consolidation-karpenter/
• Apr 2022 v0.9.0 with Pod Affinity https://aws.amazon.com/about-aws/whats-new/2022/04/amazon-eks-karpenter-v0-9-0-pod-affinity/
• Nov 2021 AWS Karpenter v0.5 https://aws.amazon.com/about-aws/whats-new/2021/11/aws-karpenter-v0-5/

Activities

• Read htps://karpenter.sh/preview/troubleshooting/
• Read about Kubernetes scheduler and Karpenter https://github.com/aws/karpenter/issues/1528
• Read Karpenter best practices: https://aws.github.io/aws-eks-best-practices/karpenter/
• Read https://aws.amazon.com/blogs/containers/managing-pod-scheduling-constraints-and-groupless-node-upgrades-with-karpenter-in-amazon-eks/

Related

• Amazon EC2 Instance types:
• Kubernetes Autoscaler
• Escalator
• EKS Best Practices Guides https://aws.github.io/aws-eks-best-practices/karpenter/
• KarpenterNode IAM Role
• https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest/examples/karpenter
• karpenter.sh: karpenter.sh/capacity-type
• kubectl logs -f -n karpenter $(kubectl get pods -n karpenter -l karpenter=controller -o name)
• Kubernetes PodDisruptionBudget (PDB)
• kubectl get all -n karpenter
• TopologyKey:
• Kubernetes scheduler
• Kubernetes Pod Topology Spread Constraints
• Kubernetes pod affinity and anti affinity
• Evict
• karpenter.sh/do-not-evict: true
• Kubernetes provisioner
• IAM OIDC provider
• Kubernetes controllers
• Amazon EC2 Auto Scaling
• kubectl get events: FailedScheduling
• Cilium

See also

• kubeclt [ get | describe | patch ] provisioner
• Karpenter, karpenter.sh, provisioners.karpenter.sh, Karpenter releases, best practices, karpenter.sh/capacity-type, karpenter.sh/discovery, kind: Provisioner, kind: AWSNodeTemplate, kubectl provisioner, TopologyKey, FailedDraining, Evict, DisruptionBlocked, Karpenter logs, controller., ttlSecondsUntilExpired, KEDA, NodePools, Kind: NodePool, Workload Consolidation, Disruption controls
• Kubernetes autoscaling, HPA, VPA, Karpenter, Kubernetes Autoscaler, kubectl autoscale, Kubernetes Metrics Server (KMS)

https://aws.amazon.com/blogs/aws/introducing-karpenter-an-open-source-high-performance-kubernetes-cluster-autoscaler/