How Do I Set Up Karpenter Auto-Scaling With GKE?

Introduction

Karpenter is an open-source Kubernetes cluster autoscaler that simplifies the process of scaling workloads in a Kubernetes environment. In this guide, we’ll walk through the steps required to set up Karpenter auto-scaling on Google Kubernetes Engine (GKE) using Pulumi. The process involves creating a GKE cluster, installing Karpenter, and configuring the necessary networking components.

Step-by-Step Explanation

To set up Karpenter auto-scaling with GKE using Pulumi, follow these detailed steps:

Create a GKE Cluster: Start by creating a GKE cluster using the gcp.container.Cluster resource. This cluster will serve as the foundation for deploying Karpenter.
Generate Kubeconfig: Obtain the credentials for the GKE cluster by generating the kubeconfig. This configuration is essential for interacting with the Kubernetes cluster.
Create a Kubernetes Provider: Use the generated kubeconfig to create a Kubernetes provider instance. This provider will facilitate the deployment of resources within the cluster.
Install Karpenter: Deploy Karpenter using a Helm chart in a dedicated namespace. This step involves creating the necessary namespace and service account for Karpenter.
Configure Networking: Set up a NetworkPolicy to manage the networking for Karpenter. This policy ensures that the appropriate ingress and egress rules are applied to the Karpenter pods.

Below is the complete Pulumi program in TypeScript to achieve this setup:

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Create a GKE cluster
const cluster = new gcp.container.Cluster("gke-cluster", {
    initialNodeCount: 1,
    minMasterVersion: "latest",
    nodeConfig: {
        machineType: "e2-medium",
    },
});

// Export the cluster name
export const clusterName = cluster.name;

// Get the GKE cluster credentials
const kubeconfig = pulumi.all([cluster.name, cluster.endpoint, cluster.masterAuth]).apply(([name, endpoint, masterAuth]) => {
    const context = `${gcp.config.project}_${gcp.config.zone}_${name}`;
    return `
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ${masterAuth.clusterCaCertificate}
    server: https://${endpoint}
  name: ${context}
contexts:
- context:
    cluster: ${context}
    user: ${context}
  name: ${context}
current-context: ${context}
kind: Config
preferences: {}
users:
- name: ${context}
  user:
    auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: gcloud
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp
`;
});

// Create a Kubernetes provider instance using the kubeconfig
const provider = new k8s.Provider("gkeK8s", {
    kubeconfig: kubeconfig,
});

// Install Karpenter using Helm chart
const karpenterNamespace = new k8s.core.v1.Namespace("karpenter", {
    metadata: { name: "karpenter" },
}, { provider });

const karpenterServiceAccount = new k8s.core.v1.ServiceAccount("karpenter", {
    metadata: {
        namespace: karpenterNamespace.metadata.name,
        name: "karpenter",
    },
}, { provider });

const karpenterChart = new k8s.helm.v3.Chart("karpenter", {
    chart: "karpenter",
    fetchOpts: {
        repo: "https://charts.karpenter.sh",
    },
    namespace: karpenterNamespace.metadata.name,
    values: {
        serviceAccount: {
            create: false,
            name: karpenterServiceAccount.metadata.name,
        },
    },
}, { provider });

// Configure networking for Karpenter
const networkPolicy = new k8s.networking.v1.NetworkPolicy("karpenter-network-policy", {
    metadata: {
        namespace: karpenterNamespace.metadata.name,
    },
    spec: {
        podSelector: {
            matchLabels: {
                "app.kubernetes.io/name": "karpenter",
            },
        },
        policyTypes: ["Ingress", "Egress"],
        ingress: [{
            from: [{
                podSelector: {
                    matchLabels: {
                        "app.kubernetes.io/name": "karpenter",
                    },
                },
            }],
        }],
        egress: [{
            to: [{
                podSelector: {
                    matchLabels: {
                        "app.kubernetes.io/name": "karpenter",
                    },
                },
            }],
        }],
    },
}, { provider });

// Export the kubeconfig
export const kubeconfigOutput = kubeconfig;

Key Points

GKE Cluster: The setup begins with creating a GKE cluster that forms the base for deploying Karpenter.
Kubeconfig: This configuration file is critical for accessing and managing the Kubernetes cluster.
Karpenter Installation: Karpenter is deployed using a Helm chart, ensuring it is correctly set up within the Kubernetes environment.
Networking Configuration: Proper network policies are established to control the traffic flow to and from the Karpenter pods.

Conclusion

By following these steps, you have successfully set up Karpenter auto-scaling with GKE using Pulumi. This configuration allows your Kubernetes cluster to efficiently manage resources and scale according to workload demands. The integration of Karpenter with GKE enhances the cluster’s ability to handle dynamic workloads, ensuring optimal performance and resource utilization.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.