Install Kubeflow on Digital Ocean Managed Kubernetes (with Terraform)

This walkthrough covers provisioning a DigitalOcean Managed Kubernetes (DOKS) cluster with Terraform and deploying Kubeflow 1.2 for machine learning pipelines. All Terraform manifests referenced below live in nsudhanva/do-k8s-challenge.

Get Started
Create a Kubernetes Cluster on Digital Ocean with Terraform
Install Kubeflow

Get Started

Install kubectl

kubectl is the standard Kubernetes CLI for inspecting cluster state and applying manifests. Follow the official Kubernetes install docs for your OS.

Install Terraform

We use Terraform to declare and provision the DigitalOcean Kubernetes cluster as code. Grab the latest binary from terraform.io/downloads.

Install doctl

doctl is DigitalOcean’s CLI. We use it here to authenticate against the DigitalOcean API and pull the cluster’s kubeconfig credentials automatically.

Install kfctl

kfctl is the CLI tool that builds and applies Kubeflow’s kustomize manifests and core components. Download the latest kfctl release and unpack the binary into a directory on your $PATH.

Create a Kubernetes Cluster on Digital Ocean with Terraform

Make sure you are on a recent Terraform 1.x release before initializing the provider.

Create main.tf to wire up the DigitalOcean provider and input token variable:

terraform {
  required_providers {
    digitalocean = {
      source  = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
  }
}

# Set the variable value in *.tfvars file
# or using -var="do_token=..." CLI option
variable "do_token" {}

# Configure the DigitalOcean Provider
provider "digitalocean" {
  token = var.do_token
}

2. Create cluster.tf to define the DOKS cluster and its autoscaling node pool:

resource "digitalocean_kubernetes_cluster" "k8s-challenge" {
  name   = "do-k8s-challenge"
  region = "nyc1"
  # Grab the latest version slug from `doctl kubernetes options versions`
  version = "1.19.15-do.0"

  node_pool {
    name       = "default"
    size       = "s-4vcpu-8gb"
    auto_scale = true
    min_nodes  = 1
    max_nodes  = 4

    tags = [
      "challenge",
      "kubeflow"
    ]

  }
}

Sizing & version notes:

Don’t undersize the nodes: Kubeflow spins up Istio, Dex, Knative, Katib, Central Dashboard, and pipeline controllers all at once. Smaller droplets (2 vCPU / 4 GB) will thrash CPU and OOM-kill pods mid-install; s-4vcpu-8gb across 2–3 nodes is a practical minimum.
Autoscaling: Enabling auto_scale (up to 4 nodes) gives the cluster headroom when you trigger multi-step Kubeflow Pipelines runs.
Kubernetes version pinning: Kubeflow 1.2 relies on older CRD API versions, so we pin 1.19.15-do.0 (or the matching supported 1.19/1.20 slug from doctl kubernetes options versions).

3. Optionally add outputs.tf so Terraform prints the cluster ID and API endpoint as soon as provisioning completes:

output "kubernetes_id" {
  description = "ID of the cluster"
  value       = digitalocean_kubernetes_cluster.k8s-challenge.id
}
output "kubernetes_host" {
  description = "The hostname of the API server for the cluster"
  value       = digitalocean_kubernetes_cluster.k8s-challenge.endpoint
}

output "kubernetes_urn" {
  description = "The uniform resource name (URN) for the Kubernetes cluster."
  value       = digitalocean_kubernetes_cluster.k8s-challenge.urn
}

output "kubernetes_created" {
  description = "Created at timestamp for the cluster"
  value       = digitalocean_kubernetes_cluster.k8s-challenge.created_at
}

Terraform configuration and setup

Run terraform init in the directory containing your .tf files to download the DigitalOcean provider plugin and initialize the lock file.
Create secret.auto.tfvars (which Terraform loads automatically) to hold your DigitalOcean Personal Access Token:

do_token = 'YOUR_DO_ACCESS_KEY';

Fix a few bottlenecks first

Two account-level limits on fresh DigitalOcean accounts will trip up a Kubeflow install if you don’t address them upfront:

Droplet and volume quotas: New accounts often cap droplet and block-storage volume counts below what Kubeflow’s persistent volumes require. Open a quick ticket via DigitalOcean cloud support to bump your droplet and volume limits (at least 3–4 nodes and 6–8 volumes).
Billing verification & student credits: If you’re testing with the GitHub Student Developer Pack, the $50 DigitalOcean credit is plenty to stand up and benchmark this cluster for several days.

Create the cluster

Generate and inspect the execution plan with terraform plan -out plan-1.out.
Apply the plan with terraform apply "plan-1.out". Provisioning the control plane and node pool takes roughly 5–10 minutes.
Confirm the cluster shows as healthy in the DigitalOcean console and copy the output kubernetes_id.

Set up .kubeconfig

Grab the cluster ID printed by Terraform.
Merge the cluster credentials into ~/.kube/config by running: doctl kubernetes cluster kubeconfig save YOUR_CLUSTER_ID

Verify the cluster

Run kubectl get namespaces to verify connectivity against the new control plane.

You should see default, kube-system, and kube-public ready to go.