ADR 3 — Software RAID scope

Status: accepted

Context#

Issue #5 asked for a way to declare a software RAID array from the corium: block, with the explicit caveat that it should only be added if it earns its place over a plain cloud-init disk setup.

There are two separate problems behind the one request.

Spare disks. Extra disks on a node, to be mirrored or striped and mounted somewhere. This can be done at first boot.

The root filesystem. By the time anything reads the corium: block, the root filesystem is deployed, mounted, and running the process doing the reading. Nothing at that point can move it onto an array. A redundant root has to be arranged by whatever laid the disk down, which means the installer.

Decision#

raid[] covers spare disks. Arrays are created by corium-agent at first boot, before k0s is installed.

A RAID root filesystem is documented as an install-time Kickstart procedure, verified on hardware, and is not modelled in the schema.

Why spare-disk RAID earns its place#

cloud-init has no RAID support. disk_setup does partitions and filesystems, and the documentation has said for years that mdadm support is anticipated. The honest alternative is therefore a runcmd block calling mdadm, and three things make that worse than a field:

• Ordering. runcmd races the kubelet. Losing that race means containerd writes its state into the directory on the root disk that the array is then mounted over. The data is invisible afterwards while still consuming the root disk, and the node reports a full disk with no apparent cause. Corium builds arrays before k0s is installed and writes RequiresMountsFor into the k0s unit, so Kubernetes refuses to start rather than run without its storage.
• Refusing to destroy data. A device already carrying a filesystem, a partition table, or another array’s metadata stops the bootstrap. mdadm --create by hand does not ask.
• Idempotency. A second bootstrap adopts an existing array rather than rebuilding it, and rewrites its own fstab line rather than appending another.

Why a RAID root is not a schema field#

Not because it does not work. It does: a bootc install onto RAID1 / and RAID1 /boot was verified on a two-disk machine, including detaching the first disk and booting from the second, with the arrays degraded and serving.

It is not a field because a field would be a lie about when the decision is taken. The corium: block is read by corium-agent at first boot, by which point the root filesystem is deployed, mounted, and running the process reading the block. There is nothing a field could do at that moment.

Three things additionally have to be done by hand at install time, none of which Anaconda does for you, and all of which are documented in docs/raid.md:

1. Build a second ESP on the mirror disk and register it with the firmware. Anaconda creates one ESP and warns that a drive failure will make the system unbootable; it is right.
2. Do that in %post using disk-level tools only — %post writes into the deployment’s /etc and /var do not survive a bootc install.
3. Add nofail to the /boot/efi line in /etc/fstab. Without it the mount is RequiredBy=local-fs.target, so losing the first disk drops the machine into emergency mode with perfectly healthy mirrored filesystems.

Fedora CoreOS collapses all three into Ignition’s boot_device.mirror, which builds the array in the initramfs and replicates /boot per disk. bootc has no equivalent. Until it does, this belongs in a procedure, not a field.

Consequences#

A node whose root disk dies is reprovisioned rather than repaired. That is the model the rest of Corium is built around: the OS is an image, the node is cattle, and a replacement boots the same digest.

Spare-disk arrays cover the case that is actually expensive — local state that is slow to rebuild — while the cluster covers the case of losing a whole node.

If bootc grows a declarative equivalent of boot_device.mirror, this decision should be revisited; the schema has room for it.

Alternatives considered#

Documentation only, no field. Ship the mdadm recipe and add no permanent key. Rejected: it leaves the ordering problem unsolved, and the ordering problem is the one that silently corrupts a node.

Support a RAID root via Kickstart as a product feature. Rejected for now, but on narrower grounds than “it does not work”. It does work, and the procedure is documented. What it is not is declarative: it needs a hand-written kickstart, a %post block, and a post-install fstab edit, and it applies only to the ISO path. Wrapping that in a Corium field would promise a uniformity the install paths do not have.

bootc-generic-growpart.service also fails permanently on an md root — it reads /sys/class/block/<name>/partition, which an md device lacks. Harmless, but it leaves every such node reporting degraded, which is a poor thing to ship as a supported configuration.

If bootc grows a declarative equivalent of boot_device.mirror, revisit both this and the decision above.