A rescue system is a live environment. When you ask for it, the provider boots
it instead of your disk. It is the usual way into a dedicated server that has
no usable remote console. It is also enough to install Corium, because the image
installs itself and the rescue only has to run it.
This page gives two routes. The rescue system decides which route you get, not
you. Route A is three commands. Route B exists because some rescue systems
cannot run a container at all. The OVH rescue system is one of them.
The machine is booted into its rescue system from the provider’s panel, and
you have a shell on it.
You know which disk is the target. Both routes overwrite one whole disk.
You have a public SSH key and the configuration that you want the node to
have. A dedicated server has no metadata service, so both routes add these by
hand.
For route B, the Corium qcow2 is on the rescue system, together with
qemu-img, sgdisk, growpart, resize2fs, setfattr and getfattr. The
last two come from the attr package. Many rescue systems do not include it,
and the OVH rescue system does not. See
downloads for the qcow2
and for how to make sure that the file you received is correct.
If the output is rootfs rootfs, the rescue runs entirely from a ramfs, and
route A cannot work. No podman flag can avoid this. The kernel refuses
pivot_root when the current root is the initial rootfs, so every container
fails with:
In that case, use route B. If the output shows a real filesystem, for example
tmpfs, ext4, or overlay, containers will run. Use route A. If route A
then fails, the cause is something else.
The OVH rescue is a ramfs. So the most common reason to read this page is a
route B case.
bootc install to-disk runs from inside the Corium image and writes a complete
disk: partition table, ESP, bootloader, and the ostree deployment. You do not
need to convert or resize anything afterwards.
Warning. Make sure that you know which disk is the target before you run
this command. --wipe destroys everything on the target disk, and you cannot
recover it. A rescue system numbers disks in its own order. That order can be
different from the order of the installed system. Also, on a machine with
several disks, the OS is usually on the small NVMe and the data is on the
large disks.
Do this step before the reboot. A dedicated server has no metadata service and
no seed device. So cloud-init finds nothing, and the node starts with no
configuration. The agent journal then shows
no corium configuration found, leaving node unconfigured. cloud-init also
creates the login user, so this node has no account for SSH. Corium ships no
root account, and its sshd drop-in sets PermitRootLogin no.
Since
ADR 11, you can recover from this
without a second visit. The node serves its enrolment route and prints a
pairing code on the console. Then cctl enroll --config can tell the node what
it is. But this still needs a console, and a dedicated server rarely gives you
a usable one. So seed the configuration here anyway. Use the API only as the
fallback if you find out later that you did not.
The cheapest fix is a kernel argument. bootc install writes it for you, so
add it to the command in step 1, not after it:
This gives Corium its configuration, but it does not create the user. To get
both, put a NoCloud seed on the installed system instead. Before cloud-init
stops, it looks for a seed in /var/lib/cloud/seed/nocloud, and ds-identify
finds it with no extra configuration.
Route B writes one, with the SELinux labels. Mount
the root filesystem that bootc install created, and do the same steps there.
Disable rescue mode in your provider’s panel and reboot. If rescue mode stays
enabled, the machine boots the rescue again, and the installed system never
runs.
This route uses the published qcow2. It contains the same installed system that
bootc install produces, laid out for a 10 GiB disk. Write it, then grow the
root partition to the size of your real disk.
Warning. Make sure that you know which disk is the target before you run
this command. qemu-img convert writes directly to the raw device. It
destroys everything on the target disk, and you cannot recover it. A rescue
system numbers disks in its own order. That order can be different from the
order of the installed system. Also, on a machine with several disks, the OS
is usually on the small NVMe and the data is on the large disks.
lsblk -o NAME,SIZE,TYPE,MODEL
A rescue system rarely has oras, and it is not worth installing it to pull
one blob. The CDN copy has a plain HTTPS URL, and the release notes publish its
digest. The bytes are the same from both sources:
curl -fsSLO https://corium.b-cdn.net/corium-0.5.0-x86_64.qcow2
sha256sum corium-0.5.0-x86_64.qcow2 # must equal the digest in the release notes
apt-get install -y qemu-utils # or the distribution's equivalentqemu-img convert -O raw -p corium-0.5.0-x86_64.qcow2 /dev/nvme0n1
sync && partprobe /dev/nvme0n1
sgdisk -e /dev/nvme0n1 # move the backup GPT to the end of the real diskgrowpart /dev/nvme0n1 4e2fsck -fp /dev/nvme0n1p4
resize2fs /dev/nvme0n1p4
Partition 4 is the root filesystem. To make sure, run
lsblk -o NAME,SIZE,FSTYPE,LABEL and find the root label. The resize keeps
the filesystem UUID. So the bootloader entries in the image still point at the
correct place.
The seed gives the node a configuration and an account for login. It must be in
place before the reboot. Mount the root filesystem and write the seed into the
/var of the deployment. In an ostree system, that is
ostree/deploy/default/var:
That corium: block has no role, on purpose. With a dedicated server, you
often do not know the details yet. You decide its disks, its name, and often
its role when you can see the machine. A document with no role holds the
bootstrap until you send a role:
If you write role: single here instead, the node builds a single-node cluster
when you claim it. The cluster gets a name derived from the machine ID and has
no array under it. You can want that, and cctl asks before it does it. But to
undo it, you must run cctl reset and wait as long as for a second install. See
configuration §3.2.
The extra groups are the only difference from the users: block on the other
install pages. adm and systemd-journal let the account read the journals,
because there can be no other way into this machine. See
Proxmox step 3 for the simpler version.
Warning. SELinux causes the most common failure here. Corium runs in
enforcing mode. A rescue system that does not know SELinux creates files with
no label, and cloud-init cannot read files with no label. A node that boots
with an unlabeled seed behaves exactly like a node with no seed. This failure
is the one most likely to cost you an afternoon.
Label the files by hand. The mkdir -p above created exactly these four paths.
So when you label them, you cover everything that the rescue system created:
cd /mnt/root/ostree/deploy/default/var/lib/cloud
for p in seed seed/nocloud seed/nocloud/meta-data seed/nocloud/user-data;do setfattr -n security.selinux -v "system_u:object_r:cloud_var_lib_t:s0""$p"donegetfattr -R -n security.selinux seed
The last command prints one line for each path, and each line is
security.selinux="system_u:object_r:cloud_var_lib_t:s0". If a path shows
No such attribute instead, cloud-init cannot read it. Before the loop runs,
all four paths show this.
/var/lib/cloud comes from the image, and it already has its label. So the
loop does not include any directory above seed. Also, the loop starts from
that directory and does not use .., because with .. it is easy to select
the wrong directory. Make sure that the label is correct. From that directory,
run getfattr -n security.selinux --only-values .. It must print the same type.
If it prints nothing, label it the same way. Label its parents too: var_lib_t
for /var/lib and var_t for /var. A directory with no label is
unlabeled_t, and cloud-init cannot go through it. So one parent with no label
causes the same failure as a user-data with no label.
On 0.3.3 and earlier, add one more file.ds-identify decides if
cloud-init runs at all. It runs as a systemd generator, before /var is
mounted. So it cannot see the seed that you wrote, and it decides that there
is nothing to do. The node then boots healthy but does nothing: no account,
no API, and only a login prompt on the console. 0.3.4 and later include the
fix. On an earlier image, write the file yourself, into the /etc of the
deployment:
Unmount everything, disable rescue mode in the provider’s panel, and
reboot. If rescue mode stays enabled, the machine boots the rescue again, and
the installed system never runs.
The OVH rescue (rescue12-customer) runs from a ramfs, so route B is the route
that works. The rescue also has no overlayfs and no pids cgroup controller.
Those two gaps cause their own errors first, and the errors send you in the
wrong direction:
What you see
Cause
'overlay' is not supported over ramfs
no overlayfs in the rescue kernel
the requested cgroup controller 'pids' is not available
The kernel was built without it. --pids-limit=0 gets past it
crun: pivot_root: Invalid argument
The real blocker: a ramfs root
Investigate the first two errors only until you reach the third.
The rescue also has no attr package. So the labels in step 2 fail with
setfattr: command not found. On this page, that is the one step where a
failure looks like success and costs you an afternoon. Run
apt-get install -y attr first. The rescue has working networking and apt.
The qcow2 contains both the removable path EFI/BOOT/BOOTX64.EFI and
EFI/fedora/shimx64.efi. So a UEFI firmware finds a bootloader without help.
An explicit entry costs nothing and removes any doubt. Add it after step 1 of
route B and before you leave rescue mode:
apt-get install -y efibootmgr
efibootmgr | grep Corium # already there? -c would add a second oneefibootmgr -c -d /dev/nvme0n1 -p 2 -L "Corium" -l '\EFI\fedora\shimx64.efi'
-c creates a new entry every time it runs. It does not update an existing
entry. So if you install a machine twice, it gets two Corium entries that point
at the same file. efibootmgr -b <n> -B deletes one. The image fixes the
partition UUIDs. So an entry that you made before a reinstall still works after
it, and usually you do not have to do it again.
Recent hardware is UEFI. To make sure for your machine, run
[ -d /sys/firmware/efi ] && echo UEFI from the rescue.
OVH gives the server its address over DHCP, and the image ships no
NetworkManager connection profile. So the interface starts without help. You
do not have to configure anything to reach the node at the address that the
panel shows.
The BLS entry includes console=ttyS0. So if the node does not come back, you
can watch the boot on the OVH IPMI serial console.