Blog #kubernetes #devops #oracle-cloud

K3s on Oracle Cloud Always Free: GitOps Kubernetes (Gateway API + Auto HTTPS)

Running a 3-node K3s cluster on Oracle Cloud Always Free with Terraform, Argo CD GitOps, Envoy Gateway (Gateway API), and automated Let’s Encrypt TLS.

Published
Updated
Reading
5 min

Live docs: https://k3s.sudhanva.me

Getting started: https://k3s.sudhanva.me/getting-started/prerequisites/

Source repo: https://github.com/nsudhanva/k3s-oracle

If you want a multi-node Kubernetes cluster for experiments, side projects, or small services without a monthly cloud bill, Oracle Cloud Infrastructure’s (OCI) Always Free tier gives you 4 Ampere A1 OCPUs and 24 GB of RAM at $0/month.

In fact, the documentation site at https://k3s.sudhanva.me is hosted on this exact cluster.

This repository provisions a 3-node K3s cluster on Ampere A1 ARM64 instances with Terraform and bootstraps Argo CD for GitOps delivery. Incoming traffic routes through Envoy Gateway using the Kubernetes Gateway API, with ExternalDNS managing Cloudflare records and cert-manager issuing Let’s Encrypt TLS certificates automatically.

What you get

  • K3s cluster (lightweight Kubernetes) on OCI Always Free compute
  • Argo CD bootstrapped for GitOps continuous delivery
  • Gateway API ingress via Envoy Gateway
  • Automatic DNS records in Cloudflare via ExternalDNS
  • Automatic TLS certificates from Let’s Encrypt via cert-manager
  • Secrets stored in OCI Vault, synced into Kubernetes via External Secrets
  • Infrastructure managed end-to-end with Terraform

Architecture

Traffic flow: User → Cloudflare DNS → OCI Network Load Balancer → Ingress/NAT node → K3s server + worker.

graph TB
    subgraph Internet
        User((User))
        CF[Cloudflare DNS]
    end

    subgraph OCI["Oracle Cloud (Always Free)"]
        NLB[Network Load Balancer]
        subgraph Public["Public Subnet"]
            Ingress[k3s-ingress<br/>NAT + Envoy]
        end
        subgraph Private["Private Subnet"]
            Server[k3s-server<br/>Control Plane]
            Worker[k3s-worker<br/>Workloads]
        end
    end

    subgraph GitHub
        Repo[(Repository)]
    end

    User -->|HTTPS| CF
    CF --> NLB
    NLB --> Ingress
    Ingress --> Server
    Ingress --> Worker
    Repo -->|GitOps| Server

Node layout (Always Free–friendly)

The cluster runs on three Ampere A1 ARM64 instances within OCI’s Always Free limits (4 OCPUs, 24GB RAM total):

Node Resources Subnet Role
k3s-ingress 1 OCPU, 6GB Public (10.0.1.0/24) NAT gateway, Envoy Gateway
k3s-server 2 OCPU, 12GB Private (10.0.2.0/24) K3s control plane, Argo CD
k3s-worker 1 OCPU, 6GB Private (10.0.2.0/24) Application workloads

How this layout fits the free-tier limits:

  • Single public choke point: k3s-ingress is the only VM exposed to the public internet and handles both NAT egress and Envoy ingress for north–south traffic.
  • Isolated private subnet: Both the K3s control plane and application worker nodes live entirely off the public internet.
  • Stable front door: OCI’s flexible Network Load Balancer sits in front of the ingress node for clean HTTPS termination and routing.
  • Low memory footprint: You get the Kubernetes Gateway API, cert-manager, ExternalDNS, and External Secrets without spending half your 24 GB RAM budget on a service mesh.

Components

Component Purpose
K3s Lightweight Kubernetes distribution
Argo CD GitOps continuous delivery
Envoy Gateway Gateway API implementation
ExternalDNS Automatic Cloudflare DNS updates
cert-manager Let’s Encrypt certificate automation
OCI Vault Secrets storage (Always Free)
External Secrets Sync Vault secrets into Kubernetes

This diagram shows how provisioning, GitOps, and the runtime services connect end-to-end:

flowchart LR
    subgraph Infra["Infrastructure"]
        TF[Terraform]
    end

    subgraph Cluster["K3s Cluster"]
        Argo[Argo CD]
        EG[Envoy Gateway]
        CM[Cert Manager]
        ED[External DNS]
    end

    subgraph External["External Services"]
        OCI[(OCI)]
        GH[(GitHub)]
        GHCR[(GHCR)]
        CFl[(Cloudflare)]
        LE[(Let's Encrypt)]
        Vault[(OCI Vault)]
    end

    TF -->|provisions| OCI
    TF -->|provisions| Vault
    TF -->|generates| GH
    GH -->|Actions| GHCR
    GH -->|syncs| Argo
    GHCR -->|images| Argo
    Argo -->|deploys| EG
    Argo -->|deploys| CM
    Argo -->|deploys| ED
    Argo -->|applies| ES[ExternalSecret]
    Vault -->|syncs to| ES
    ES -->|creates| Secret[K8s Secret]
    CM -->|certificates| LE
    ED -->|DNS records| CFl

OCI Always Free resources used

Resource Free Limit Usage
Ampere A1 Compute 4 OCPUs, 24 GB RAM 4 OCPUs, 24 GB
Object Storage 20 GB ~1 MB (Terraform state)
Vault Secrets 150 secrets ~10 secrets
Vault Master Keys 20 key versions 1 key
Flexible NLB 1 instance 1 instance (ingress)

Prerequisites

You need:

  • An OCI account with Always Free eligibility
  • A Cloudflare account with a managed domain
  • A GitHub account and Personal Access Token (PAT)
  • Terraform installed locally

Quick Start

1) Create configuration

Create tf-k3s/terraform.tfvars:

tenancy_ocid         = "ocid1.tenancy.oc1..."
user_ocid            = "ocid1.user.oc1..."
fingerprint          = "xx:xx:xx..."
private_key_path     = "/path/to/oci_api_key.pem"
region               = "us-ashburn-1"
compartment_ocid     = "ocid1.compartment.oc1..."

ssh_public_key_path  = "/path/to/ssh_key.pub"
cloudflare_api_token = "your-cloudflare-token"
cloudflare_zone_id   = "your-zone-id"
domain_name          = "k3s.example.com"
acme_email           = "admin@example.com"

git_repo_url         = "https://github.com/your-user/k3s-oracle.git"
git_username         = "your-github-username"
git_email            = "your-email@example.com"
git_pat              = "ghp_..."

k3s_token             = "your-random-secure-token"
argocd_admin_password = "your-secure-password"

2) Deploy infrastructure + bootstrap the cluster

cd tf-k3s
terraform init
terraform apply

3) Push GitOps manifests

Terraform renders the cluster-specific Argo CD manifests into argocd/. Commit and push them so Argo CD can begin reconciling the cluster:

git add argocd/
git commit -m "Configure cluster manifests"
git push

4) Verify

Initial bootstrap takes a few minutes while K3s and Argo CD come up. Once it finishes, jump through the ingress bastion to verify that all Argo CD applications are synced:

ssh -J ubuntu@<ingress-ip> ubuntu@10.0.2.10 "sudo kubectl get applications -n argocd"

CI/CD

GitHub Actions workflows run the lint checks and the deployment.

Workflow Trigger Purpose
lint.yml Pull requests Run pre-commit hooks (markdownlint, yamllint, tflint)
docker-publish.yml Push to main (docs/) Build Docker image, push to GHCR, and restart the docs pod

GitHub Secrets required

For automatic docs deployment, add these secrets at Settings → Secrets → Actions:

Secret Value
SSH_PRIVATE_KEY Contents of your SSH private key (same key used in ssh_public_key_path)
INGRESS_IP Public IP of the ingress node (from terraform output ingress_public_ip)

Local development

pre-commit install
pre-commit run --all-files

Documentation

Open the live cluster documentation

To view and edit the docs locally:

cd docs
bun install
bun start

Related work

Production ML context

See how this topic connects to production ML systems, infrastructure, and inference.