WireGuard overlay

Corium has no native field for a host WireGuard interface yet. The design is decided but not yet built (ADR 6) — and for overlays that are not the cluster’s own transport it stays optional even once it ships. Until the field lands you build the interface the way you build anything the corium: block does not cover: with the escape hatches every node always has, write_files and runcmd. This page is that recipe.

It is worth being clear about what this earns you and what it does not, because the honest answer shapes when to use it. A WireGuard overlay gives every node an address in one private range, so hosts that share no network — different sites, different providers — can still reach each other over an encrypted link. That is a host concern, separate from the CNI: k0s can already encrypt pod-to-pod traffic, but that assumes the hosts underneath can find each other, which across providers they cannot. The overlay is what lets them.

Three things this recipe cannot do cleanly, all covered below in full. It runs only where cloud-init runs, so a node configured without a cloud-init datasource — bare metal, PXE, an appliance reading /etc/corium/config.yaml — never executes it at all (section 1). It puts the private key in cleartext in your instance metadata (section 5). And it gives you no reliable way to make the kubelet register the overlay address as the node’s address (section 3). The first rules the recipe out entirely off-cloud; the other two are the reason the native field was chosen: if the overlay is meant to be the cluster’s own transport, read section 3 before you rely on it.

1. What you need before you start#

The module is in the image; the tools are not — you add them. The kernel carries wireguard.ko present-but-unloaded (it loads on demand the first time wg-quick brings an interface up), but wireguard-tools is not in the base image, so wg, wg-quick and the wg-quick@.service template are absent. You cannot add them at boot — /usr is read-only and cloud-init’s packages: is disabled on this OS — so it is an image change: derive an image and install the one package, following building your own image for the signing that lets nodes accept the upgrade.

FROM ghcr.io/corium-os/corium:0.2.0

# wg, wg-quick, and the wg-quick@.service template the recipe enables.
RUN dnf install -y --setopt=install_weak_deps=False wireguard-tools \
    && dnf clean all \
    && rm -rf /var/cache/* /var/lib/dnf /var/log/dnf* /var/log/hawkey.log

RUN bootc container lint --fatal-warnings

Everything below assumes your nodes run an image with wireguard-tools present. The native field (ADR 6) will make Corium ship the package by default, present-but-inert the way it ships sshd; until that lands, this one line is yours to add.

This recipe needs cloud-init. write_files and runcmd are cloud-init modules; they run only on a node that has a cloud-init datasource. On bare metal, PXE, or an appliance that carries its corium: block in /etc/corium/config.yaml or on the kernel command line, cloud-init may not run at all, and then none of this executes. That is the case the native field is meant for and this recipe is not: a corium: field is read from the whole configuration source chain, where a runcmd is read from cloud-init alone. If your nodes are off-cloud, this page will not help you — the field is what you want, and saying so on the issue is how it gets built.

2. A plain host overlay#

This is the clean case: an encrypted network between hosts, used for something other than the cluster’s own control-plane traffic — cross-host database replication, a private admin network, a service that should never touch the public internet. Ordering against k0s does not matter here, so the recipe is just the interface.

#cloud-config
#
# The WireGuard config carries this node's private key. Anything that can read
# this instance's metadata can read it. That is the cost of doing this in
# cloud-init rather than through a secret reference; see section 5.
write_files:
  - path: /etc/wireguard/wg0.conf
    permissions: '0600'
    owner: root:root
    content: |
      [Interface]
      Address = 10.10.0.2/24
      ListenPort = 51820
      PrivateKey = REPLACE_WITH_THIS_NODES_PRIVATE_KEY

      [Peer]
      PublicKey = REPLACE_WITH_PEER_PUBLIC_KEY
      Endpoint = gateway.example.com:51820
      AllowedIPs = 10.10.0.0/24
      # Needed when this node sits behind NAT and must keep the path open.
      PersistentKeepalive = 25

runcmd:
  - [ systemctl, enable, --now, wg-quick@wg0.service ]

enable rather than start is the point: wg-quick@wg0.service comes back on every reboot, not just this boot. /etc/wireguard is the only place wg-quick reads from, so the config has to live there; it is written 0600 because it holds the key.

Confirm it came up with wg show on the node.

3. An overlay used as the cluster transport#

If the cluster itself runs over the overlay — nodes joining across sites, the API server reachable only on the private range — two more things have to be true, and they are where this stops being a copy-paste job.

The interface has to be up before k0s starts, on every boot. On the first boot cloud-init’s runcmd runs during cloud-final.service, and corium-bootstrap.service — which installs and starts k0s — is ordered after it, so the overlay is already up. On later boots there is no such guarantee unless you write it down: order the k0s unit after the interface. The unit is k0sworker.service on a worker and k0scontroller.service on a controller, and it is installed by corium-agent at runtime, but a drop-in placed here is picked up when the unit appears:

write_files:
  # k0scontroller.service on a controller or controller+worker node.
  - path: /etc/systemd/system/k0sworker.service.d/10-wireguard.conf
    permissions: '0644'
    content: |
      [Unit]
      After=wg-quick@wg0.service
      Requires=wg-quick@wg0.service

Requires and not just After so that a node whose overlay failed to come up does not start k0s and half-join a cluster on the wrong address — the same posture Corium takes for a RAID mount.

The node has to register its overlay address, and here the escape hatch runs out. For a controller, point clients and joining nodes at the overlay by setting the endpoint in the corium: block — this reaches the API server’s externalAddress and its certificate SANs, which is supported and enough for what talks to the control plane:

corium:
  role: controller+worker
  cluster:
    name: overlay
    endpoint: 10.10.0.1          # the overlay address, not the public one

What no Corium knob reaches is the kubelet’s own node address — the INTERNAL-IP in kubectl get node -o wide. Corium only pins it for an HA controller, and k0s takes it as a command-line kubelet flag that neither the k0s.patch escape hatch nor a worker profile can set. Left alone, the kubelet picks the address on the interface holding the default route, which is your public NIC, not wg0. The node then looks healthy and is unreachable from across the overlay for logs, exec, port-forward and metrics — the failure this project wrote internal/bootstrap/nodeip.go to prevent for the VRRP virtual IP.

Your options, in order of how much they cost:

• Check, then decide. Run kubectl get node -o wide after the node joins. If it registered the overlay address anyway — which happens when the overlay is the routed path — you are done.
• Route the default through the tunnel. Set AllowedIPs = 0.0.0.0/0 (and let wg-quick manage the route) so the kubelet’s auto-detection picks the overlay address. This sends all the node’s traffic through WireGuard, which is often more than you wanted and worth doing deliberately.
• Wait for the field. Making the overlay address the registered node address, before k0s, without routing everything through it, is exactly what the wireguard: field exists to do, and it is the one part of this that a recipe cannot do well.

4. Reboots and re-runs#

corium-agent bootstraps a node once and never again — the marker in /var/lib/corium sees to that. The overlay is not the agent’s; it is wg-quick@wg0.service, enabled, and it comes up on its own on every boot. That is why the recipe enables a service rather than running wg from runcmd: a runcmd is a first-boot event, and an interface that only exists on first boot is an interface that vanishes at the first reboot.

/etc/wireguard/wg0.conf lives in /etc, which OSTree three-way merges on upgrade. An edit you make on the node by hand survives, but it also means a change you ship in a later image will not reach a file the node has already diverged from — the normal /etc contract.

5. The private key#

The recipe above writes the private key into the cloud-config document, which means it sits in cleartext wherever that document does: the seed ISO, the provider’s user-data, the metadata service every process on the node can query. Corium keeps join tokens, VRRP passwords and operator CAs out of that place on purpose, through secret references resolved at boot — and a WireGuard key is no less a secret than any of them.

There is no way to close this gap from inside cloud-init alone, because the tool that reads the key, wg-quick, reads it from a file and knows nothing of secret stores. The nearest you can do by hand is fetch the key at boot from somewhere authenticated and assemble the config before bringing the interface up — which is reimplementing, less carefully, the secret handling the native field would give you. If the key not being in metadata matters to you, that is a reason to want the field rather than this recipe.

At the very least: generate a distinct key per node, keep the metadata that carries it as private as the platform allows, and rotate it if the metadata is ever exposed.

The mistakes, in the order people make them#

Reaching for a runcmd on an off-cloud node. No cloud-init datasource means no write_files and no runcmd, so on bare metal or PXE this recipe silently does nothing. That is not a bug to work around; it is the boundary of the recipe. Use the native field there.

start instead of enable. The overlay works until the first reboot, then silently does not, and the node drops off the network with nothing in the journal pointing at why.

Assuming the node registered the overlay address. It usually did not. Section 3 — check with kubectl get node -o wide, do not assume.

The key in a shared image. A private key baked into a derived image, or into one seed ISO reused across nodes, is the same key on every node. Generate one per node and deliver it as per-node metadata, not as part of the image.

A hostname endpoint that only resolves over the overlay. If a peer’s Endpoint is a name, and the DNS that resolves it is itself reachable only across the tunnel, the interface cannot come up. Use an address for the endpoint that bootstraps the overlay.

When to prefer the native field#

Use this recipe for an overlay that is not the cluster’s transport, on a node that has cloud-init anyway, or to try the idea out before the field lands. Prefer the native field — once it ships; say so on the issue if you need it sooner — when the overlay is the cluster transport, or when the node has no cloud-init datasource. That is where the recipe’s three sharp edges are: nodes with no cloud-init never run it, the kubelet registers the wrong address, and the private key sits in metadata. Those three are what the field is designed to own, and what a recipe cannot.