Bare Metal Kubernetes Homelab Setup

Built around ArgoCD, Tailscale, Envoy Gateway, Longhorn, Vault, and Cilium.

This is the blueprint for my self-hosted bare-metal Kubernetes platform running on Ubuntu 24.04 LTS. Ansible handles initial node provisioning, and ArgoCD takes over from there to reconcile infrastructure and applications through GitOps. Ingress runs over Tailscale into Envoy Gateway via the Kubernetes Gateway API, Vault and External Secrets manage credentials, and Longhorn provides replicated block storage across nodes. Before touching physical hardware, you can spin up the entire stack locally inside Multipass VMs to rehearse changes end to end.

GitHub Repository: https://github.com/nsudhanva/homelab

Table of contents

• Features
• Architecture
  • Core systems map
  • Platform services map
  • Network topology
• Quick start
  • Bare metal bring-up
  • From scratch flow
  • Local rehearsal
• Documentation
• Repository layout
• Applications
• GitOps model
• Operations
• Conventions
• Monitoring topology
• Maintainers

Features

• Multi-node bare-metal Kubernetes with kubeadm and Cilium
• GitOps-managed infrastructure and apps via ArgoCD ApplicationSets
• Tailscale Gateway API ingress with split-horizon DNS
• Vault + External Secrets for centralized secret management
• Longhorn storage, Prometheus monitoring, and optional GPU plugins
• Automated image updates with ArgoCD Image Updater
• Metrics Server for resource usage in Headlamp
• ExternalDNS automation for Gateway API routes
• Envoy Gateway data plane for Gateway API

Architecture

Core systems map

flowchart LR
  subgraph Repo["Homelab Git repo"]
    Ansible["ansible/"]
    Bootstrap["bootstrap/"]
    Infra["infrastructure/"]
    Apps["apps/"]
  end

  subgraph Nodes["Ubuntu 24.04 nodes"]
    Prep["Ansible provisioning"]
    Kubeadm["kubeadm init/join"]
  end

  subgraph Cluster["Kubernetes cluster"]
    Argo["ArgoCD + ApplicationSets"]
    Workloads["Infra + apps"]
  end

  Ansible --> Prep
  Prep --> Kubeadm
  Kubeadm --> Argo
  Bootstrap --> Argo
  Infra --> Argo
  Apps --> Argo
  Argo --> Workloads

Platform services map

flowchart TB
  subgraph Edge["Edge & ingress"]
    Tailscale["Tailscale Gateway API"]
    Envoy["Envoy Gateway"]
    DNS["ExternalDNS + split-horizon CoreDNS"]
  end

  subgraph Platform["Platform services"]
    Vault["Vault"]
    ESO["External Secrets"]
    Longhorn["Longhorn"]
    Metrics["Prometheus + Grafana"]
  end

  subgraph Apps["User apps"]
    Headlamp["Headlamp"]
    Docs["Docs"]
    Homer["Homer"]
    Media["Jellyfin + Filebrowser"]
  end

  Tailscale --> Envoy
  DNS --> Envoy
  Envoy --> Apps
  Vault --> ESO
  ESO --> Apps
  Longhorn --> Apps
  Metrics --> Apps

Network topology

flowchart LR
  Public["Public DNS"]
  Tailnet["Tailscale tailnet"]
  ExternalDNS["ExternalDNS"]
  Gateway["Tailscale Gateway API"]
  Envoy["Envoy Gateway"]
  Routes["HTTPRoutes"]
  Services["Cluster Services"]
  CoreDNS["CoreDNS rewrite"]

  ExternalDNS --> Public
  Public --> Gateway
  Tailnet --> Gateway
  Gateway --> Envoy
  Envoy --> Routes
  Routes --> Services
  CoreDNS --> Services

Quick start

Bare metal is the primary target, though I strongly recommend running the local Multipass VM flow first whenever you’re testing CNI or bootstrap changes.

Bare metal bring-up

Bringing up the cluster takes three stages:

1. Provision the Ubuntu nodes with Ansible.
2. Bootstrap kubeadm and Cilium on the control plane.
3. Hand cluster reconciliation over to ArgoCD.

nano ansible/inventory/hosts.yaml
ansible-playbook -i ansible/inventory/hosts.yaml ansible/playbooks/provision-cpu.yaml

sudo kubeadm init --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

CILIUM_VERSION=$(grep -E "cilium_version:" ansible/group_vars/all.yaml | head -n 1 | awk -F'"' '{print $2}')
cilium install --version "$CILIUM_VERSION" --values infrastructure/cilium/values.cilium

kubectl apply -k bootstrap/argocd
kubectl wait --for=condition=available --timeout=600s deployment/argocd-server -n argocd
kubectl apply -f bootstrap/root.yaml

Guided flow: https://homelab.sudhanva.me/how-to/from-scratch/ and https://homelab.sudhanva.me/tutorials/

From scratch flow

flowchart TB
  subgraph Workstation["Workstation"]
    Inventory["ansible/inventory/hosts.yaml"]
    Vars["ansible/group_vars/all.yaml"]
    Playbooks["ansible/playbooks/*"]
  end

  subgraph Nodes["Ubuntu 24.04 nodes"]
    OS["OS baseline + containerd"]
    Kubelet["kubelet + kubeadm"]
  end

  subgraph ControlPlane["Control plane"]
    Init["kubeadm init"]
    Kubeconfig["/etc/kubernetes/admin.conf"]
    CNI["Cilium install"]
    Argo["ArgoCD bootstrap"]
  end

  subgraph GitOps["GitOps reconciliation"]
    AppSets["ApplicationSets"]
    InfraApps["infra-* apps"]
    UserApps["app-* apps"]
  end

  Inventory --> Playbooks
  Vars --> Playbooks
  Playbooks --> OS
  OS --> Kubelet
  Kubelet --> Init
  Init --> Kubeconfig
  Kubeconfig --> CNI
  CNI --> Argo
  Argo --> AppSets
  AppSets --> InfraApps
  AppSets --> UserApps

Local rehearsal

Use Multipass to validate the full flow on your workstation.

./scripts/local-cluster.sh up

For a rehearsal with fewer resources:

WORKER_COUNT=0 VM_CPUS=2 VM_MEMORY=3G VM_DISK=12G CILIUM_HUBBLE_ENABLED=false ./scripts/local-cluster.sh up

To remove the local cluster:

./scripts/local-cluster.sh down

Documentation

Docs site: https://homelab.sudhanva.me

To build the docs locally:

cd docs
npm ci
npm start

Start with these pages:

• Start from scratch: https://homelab.sudhanva.me/how-to/from-scratch/
• Prereqs and system prep: https://homelab.sudhanva.me/tutorials/prerequisites/
• GitOps model: https://homelab.sudhanva.me/explanation/automation-model/
• Infra catalog: https://homelab.sudhanva.me/reference/infrastructure-components/
• App catalog: https://homelab.sudhanva.me/reference/applications/

Repository layout

ansible/          Node provisioning
bootstrap/        ArgoCD bootstrap and ApplicationSets
infrastructure/   Cluster components managed by ArgoCD
apps/             User workloads managed by ArgoCD
clusters/         Cluster-specific overrides
scripts/          Automation helpers
docs/             Docusaurus documentation

Applications

GitOps model

ArgoCD uses ApplicationSets to reconcile everything under infrastructure/ and apps/. Once bootstrap finishes, avoid running manual kubectl apply commands against the cluster and let Git remain the single source of truth.

To add a new application:

1. Create apps/<app>/app.yaml to define the ArgoCD application name, path, and target namespace.
2. Drop the Kubernetes manifests into the same directory.
3. Add a kustomization.yaml if you want ArgoCD Image Updater to commit image tag overrides automatically.

How automated image updates work:

• ArgoCD Image Updater writes .argocd-source-<app>.yaml override files directly into the application directory.
• Because those files aren’t standard Kubernetes manifests, kubeconform skips them during CI validation.

Operations

Day-to-day health checks:

kubectl get nodes
kubectl get pods -A
kubectl get apps -n argocd

Run the pre-commit suite locally before pushing manifest changes:

pre-commit run --all-files

Conventions

• ApplicationSets automatically discover directories and generate infra-* and app-* ArgoCD applications.
• Each application directory keeps its app.yaml metadata alongside its Kubernetes manifests.

Monitoring topology

flowchart TB
  subgraph CRDs["Prometheus Operator CRDs"]
    CRD["monitoring.coreos.com/*"]
  end

  subgraph Stack["Monitoring stack"]
    Prometheus["Prometheus"]
    Alertmanager["Alertmanager"]
    Grafana["Grafana"]
  end

  subgraph Sources["Metrics sources"]
    ServiceMonitors["ServiceMonitors"]
    NodeExporter["node-exporter"]
    KSM["kube-state-metrics"]
    Apps["App metrics"]
  end

  CRD --> Prometheus
  CRD --> Alertmanager
  CRD --> Grafana
  ServiceMonitors --> Prometheus
  NodeExporter --> Prometheus
  KSM --> Prometheus
  Apps --> ServiceMonitors
  Prometheus --> Alertmanager
  Prometheus --> Grafana

Maintainers

• Sudhanva Narayana
• Maanasa Narayan