# ADR 3 — Software RAID scope Status: accepted ## Context [Issue #5](https://github.com/Corium-OS/Corium/issues/5) asked for a way to declare a software RAID array from the `corium:` block, with the explicit caveat that it should only be added if it earns its place over a plain cloud-init disk setup. There are two separate problems behind the one request. **Spare disks.** Extra disks on a node, to be mirrored or striped and mounted somewhere. This can be done at first boot. **The root filesystem.** By the time anything reads the `corium:` block, the root filesystem is deployed, mounted, and running the process doing the reading. Nothing at that point can move it onto an array. A redundant root has to be arranged by whatever laid the disk down, which means the installer. ## Decision `raid[]` covers spare disks. Arrays are created by `corium-agent` at first boot, before k0s is installed. A RAID root filesystem is documented as an install-time Kickstart procedure, verified on hardware, and is not modelled in the schema. ## Why spare-disk RAID earns its place cloud-init has no RAID support. `disk_setup` does partitions and filesystems, and the documentation has said for years that mdadm support is anticipated. The honest alternative is therefore a `runcmd` block calling `mdadm`, and three things make that worse than a field: - **Ordering.** `runcmd` races the kubelet. Losing that race means containerd writes its state into the directory on the root disk that the array is then mounted over. The data is invisible afterwards while still consuming the root disk, and the node reports a full disk with no apparent cause. Corium builds arrays before k0s is installed and writes `RequiresMountsFor` into the k0s unit, so Kubernetes refuses to start rather than run without its storage. - **Refusing to destroy data.** A device already carrying a filesystem, a partition table, or another array's metadata stops the bootstrap. `mdadm --create` by hand does not ask. - **Idempotency.** A second bootstrap adopts an existing array rather than rebuilding it, and rewrites its own fstab line rather than appending another. ## Why a RAID root is not a schema field Not because it does not work. It does: a bootc install onto RAID1 `/` and RAID1 `/boot` was verified on a two-disk machine, including detaching the first disk and booting from the second, with the arrays degraded and serving. It is not a field because a field would be a lie about *when* the decision is taken. The `corium:` block is read by `corium-agent` at first boot, by which point the root filesystem is deployed, mounted, and running the process reading the block. There is nothing a field could do at that moment. Three things additionally have to be done by hand at install time, none of which Anaconda does for you, and all of which are documented in `docs/raid.md`: 1. Build a second ESP on the mirror disk and register it with the firmware. Anaconda creates one ESP and warns that a drive failure will make the system unbootable; it is right. 2. Do that in `%post` using disk-level tools only — `%post` writes into the deployment's `/etc` and `/var` do not survive a bootc install. 3. Add `nofail` to the `/boot/efi` line in `/etc/fstab`. Without it the mount is `RequiredBy=local-fs.target`, so losing the first disk drops the machine into emergency mode with perfectly healthy mirrored filesystems. Fedora CoreOS collapses all three into Ignition's `boot_device.mirror`, which builds the array in the initramfs and replicates `/boot` per disk. bootc has no equivalent. Until it does, this belongs in a procedure, not a field. ## Consequences A node whose root disk dies is reprovisioned rather than repaired. That is the model the rest of Corium is built around: the OS is an image, the node is cattle, and a replacement boots the same digest. Spare-disk arrays cover the case that is actually expensive — local state that is slow to rebuild — while the cluster covers the case of losing a whole node. If bootc grows a declarative equivalent of `boot_device.mirror`, this decision should be revisited; the schema has room for it. ## Alternatives considered **Documentation only, no field.** Ship the mdadm recipe and add no permanent key. Rejected: it leaves the ordering problem unsolved, and the ordering problem is the one that silently corrupts a node. **Support a RAID root via Kickstart as a product feature.** Rejected for now, but on narrower grounds than "it does not work". It does work, and the procedure is documented. What it is not is *declarative*: it needs a hand-written kickstart, a `%post` block, and a post-install `fstab` edit, and it applies only to the ISO path. Wrapping that in a Corium field would promise a uniformity the install paths do not have. `bootc-generic-growpart.service` also fails permanently on an md root — it reads `/sys/class/block//partition`, which an md device lacks. Harmless, but it leaves every such node reporting `degraded`, which is a poor thing to ship as a supported configuration. If bootc grows a declarative equivalent of `boot_device.mirror`, revisit both this and the decision above.