Concepts

This page explains the model that the rest of the documentation uses. Read it once, and the quick start and the reference will be easier to follow.

The operating system is an image#

On most Linux systems, you build the system on the machine. You install it, add packages, edit the configuration, and apply updates. Two machines that start identical become different over time, and neither matches the plan.

Corium is built once, as an OCI image, with an ordinary Containerfile. That image is the operating system. You push it to a registry, address it by digest, and install it onto machines. Each machine starts identical to the others because each one gets the same image.

You already have the tools for it. You inspect the image with podman, scan it with your scanner, and promote it when you move a tag. Two builds share most of their layers, so an upgrade transfers only the difference. On a 2.3 GB image, we measured 214 MB.

The mechanism is bootc. It boots a container image as a system on top of OSTree.

The filesystem contract#

Most mistakes in this type of system come from a wrong model of the filesystem.

                  ┌──────────────── the image ────────────────┐
   boot image A   │  /usr    binaries, systemd units, k0s     │  read-only
                  └───────────────────────────────────────────┘
                            │
      upgrade               │  /usr replaced wholesale
      to image B            ▼
                  ┌──────────────── the machine ──────────────┐
                  │  /etc    three-way merged: your edits win │  writable
                  │  /var    untouched: k0s state, etcd, logs │  writable
                  └───────────────────────────────────────────┘

/usr is the image. Nothing writes to it at runtime. When you boot a new image, all of /usr is replaced. The k0s binary in /usr is the default Kubernetes version of the image. A signed system extension can put another version over it, as shown below.

/etc is merged. OSTree compares the defaults of the image with the edits on the machine and combines them. If you edit a file, that file stops following the version in the image. This is useful, but you must remember it.

/var is yours. The image never changes it. For this reason, upgrades are safe and rollbacks are partial. When you roll back the OS, the data that the cluster wrote stays.

One result of this surprises people. You cannot ship data in /var by putting it into the image. The installer writes /var once, at install time, and never again. You declare state directories in tmpfiles.d instead.

First boot, exactly once#

A generic image must become one specific node. corium-agent does this, and it runs once.

  ┌─ configuration sources, most specific first ─┐
  │  /etc/corium/config.yaml                     │   an operator, this machine
  │  cloud-init                                  │   the platform
  │  corium.config= on the kernel command line   │   PXE
  │  /usr/share/corium/config.yaml               │   a default in the image
  └──────────────────┬───────────────────────────┘
                     │  first source that answers wins
                     ▼
              ┌─────────────┐
              │ corium-agent│  validate everything before touching anything
              └──────┬──────┘
                     │
        ┌────────────┼─────────────┬──────────────────┐
        ▼            ▼             ▼                  ▼
   hostname     /etc/k0s/     k0s install        marker written to
   (stable)     k0s.yaml      + start            /var/lib/corium
                                                        │
                        next boot ──────────────────────┘
                        sees the marker and does nothing

“Once” is the important word. If cluster bootstrap runs again on a node that already joined, it destroys data. For this reason, the agent writes the marker last and reads it first. The marker is in /var, so it stays after upgrades.

The agent stops early and reports clearly. It validates the whole configuration before it changes anything, and it reports all problems at the same time. A node that stops with a reason in the journal is better than a node that joins a cluster halfway and looks healthy.

Two more details come from the same reasoning. First, the agent derives the name of a node from the machine ID. It does not make a random name, because a random name registers a new node at each reboot and leaves the old one behind. Second, if a source fails for a reason other than “absent”, the search stops. Your configuration can be only unreachable. If the search then continues to a default in the image, the node can join the wrong cluster.

What a machine does on its very first boot#

The document that the chain found decides which of these happens. Read this table once. If you confuse the last two rows, you discover the difference with a cctl reset.

Be careful with the fourth row. A claim releases the bootstrap. So cctl enroll against such a node is not a reservation. It starts a cluster, and to undo it you must run cctl reset. The node tells you this and asks before it continues.

With the fifth row, one identical cloud-config describes a whole fleet. The document carries no secrets and names no machine:

#cloud-config
corium:
  api:
    enabled: true

All machine-specific configuration then comes over the API, with cctl enroll --config or cctl apply. api.awaitConfig: true says the same thing explicitly. Use it to ask for the wait in a document that names a role.

Configuration is cloud-init, not a new API#

The corium: block is part of an ordinary cloud-config. Every cloud, hypervisor and PXE setup already supports cloud-config. role is the only required field. A node that waits for someone to tell it what it is can leave out even role, as described above.

Everything that the schema does not model is available through k0s.patch, which Corium applies verbatim. The project follows this rule: you can reach every k0s feature, but some features have no shorter name. See feature support.

There is a management API. It does not contradict the heading of this section, because the two answer different questions. Cloud-init describes what a machine will become, once, before it exists. The API tells you what a running machine is. It also does the few tasks that an operator needs after that: read the node, restart k0s, upgrade it, and take it out of service. The API cannot write a corium: block. If a node needs a different configuration, you reprovision it. You do not edit it.

The API is off unless the configuration of a node asks for it. See cctl, and ADR 4 for the reasons for its design.

Kubernetes ships with the OS#

k0s is a single static binary with no host dependencies. It keeps its state under /var/lib/k0s. This matches the filesystem contract: the binary is in /usr, and the state is in /var.

The image carries a Kubernetes version, but that version is not fixed. The binary in /usr is a floor. A node runs it when the configuration names no version, and it is what the node falls back to. A signed system extension can put another version over it, from the window of versions that the image supports. So the OS and Kubernetes move on separate axes. A new image upgrades the OS. kubernetes.version or cctl k8s upgrade changes Kubernetes. Neither one forces the other to change, and neither one writes in place. See ADR 10.

What Corium is not#

Corium does not manage fleets, fork k0s, or invent a configuration language. These limits are intentional. The project is an integration with strong opinions, and much of its value comes from what it does not do.

The management API is not an exception to the first limit. It is one daemon per node, and it answers only for that node. It has no registry, no inventory, and no reconcile loop. A cluster upgrade is a cctl loop over addresses that you supply, and it runs on your machine. The operator controls the order, not a node. The reasons for each limit are in feature support.

Next#

• Quick start: use this model on a real node
• Examples: a complete document for each common node shape
• Configuration reference: every field, and what Corium does with it
• Comparison: how this model is different from Talos, Kairos and others