# ADR 4 — Management API<no value>

<!-- Generated by website/sync-docs.py from docs/adr/0004-management-api.md. Edit that file, not this one. -->

Status: accepted

## Context

[Issue #2](https://github.com/Corium-OS/Corium/issues/2) asks for an API to
manage the operating system. The discussion on it converged quickly on the two
things worth having first — reading a node's state and driving its upgrade —
and just as quickly on the two questions that actually decide the design: what
such an API is allowed to do, and how a node learns which operator to obey.

Today the answer to both is SSH. That is worth stating plainly, because an
immutable OS makes SSH a poor fit in a way that is easy to miss. The shell you
land in is a shell over a system where almost nothing you type persists: `dnf`
is absent, `/usr` is read-only, and the changes that do survive land in `/etc`
and `/var` where they silently diverge from the image on the next upgrade. What
an operator actually wants — the booted image digest, the last hundred lines of
the k0s journal, a staged upgrade, a drain and a reboot — are a handful of
operations, and SSH grants root to get at them.

Two of Corium's stated non-goals sit close enough to this request to be worth
disarming before going further.

**"Corium is not a fleet-management control plane."** It is not, and this does
not make it one. What is proposed is a daemon on each node that answers
questions about that node and acts on that node. There is no registry, no
inventory, no desired state, nothing that reconciles, and no server anywhere
holding a list of machines. A cluster-wide upgrade is a `cctl` loop over
addresses the operator supplied, running on the operator's laptop. If that
sequencing should one day be driven by a controller, the controller will be
somebody else's, and it will talk to this same API.

**"A bespoke configuration API" is out of scope.** It stays out of scope for a
node in service: this API will not rewrite the `corium:` block of a machine
that has bootstrapped, and a node that needs different configuration is
reprovisioned. Provisioning remains cloud-init's job (decisions 6 and 7), and
the API covers day two, after the machine exists.

*Amended.* A node that has **not** bootstrapped is not yet a machine in
service, and refusing it a configuration left maintenance mode unable to serve
the case it was built for. See "Configuration, before a node is a node" below.

## Decision

Corium grows `corium-apid`, a Go daemon shipped in `/usr`, serving JSON over
HTTP and mutual TLS on `7443/tcp`, on the standard library alone. The client is
`cctl`. Trust is anchored in an
**operator CA** whose certificate the node is given, and which signs the client
certificates the node will accept.

The daemon is off unless a node is told to run it, and a node that has not yet
been claimed by an operator is not a cluster member. Those two are as much a
part of this decision as the API surface itself.

### What it manages

Four surfaces, every one of them scoped to the machine answering the call.

**Node state**, read-only: role, hostname, machine ID, the booted and staged
bootc images with their digests, the k0s version, the greenboot verdict,
uptime, kernel. This is the endpoint that makes the other three safe to use,
because it is how an operator finds out what a node is before doing anything
to it.

**Services and journals**: the state of the units Corium owns and of k0s;
restarting k0s; reading the journal for a named unit over a bounded range, with
follow; `dmesg`. Units are named from an allowlist rather than passed through,
so that the API is not a way to start arbitrary systemd units.

**Upgrades**, the mechanism issue #2 spends most of its length on: stage an
image, apply it, roll back. The node refuses an image its own signing policy
would accept unsigned, so that a typo cannot rebase a cluster node onto
Silverblue. Health gating and the one-node-at-a-time roll-out live in `cctl`,
which knows about the other nodes; the daemon only ever knows about its own.

An earlier version of this record said the image's *labels* would be checked as
well. They are not, for two reasons found while building it. A Corium node
ships no skopeo, no podman and no jq — a container engine on the host is
deliberately absent, because k0s supervises its own — and bootc's status output
carries no labels, so there is nothing on the machine that can read them.
More importantly, a label is not worth reading: anybody can write
`LABEL org.opencontainers.image.title="Corium"`, so it catches a typo and
nothing else.

The signing policy catches the typo *and* the attacker, needs no tooling, and
is enforced by the runtime at pull time regardless. It is also the right shape
for the derived images Corium supports: an operator who builds their own adds
their repository and key to `/etc/containers/policy.json`, and their image
becomes acceptable because they said so rather than because it claimed to be
Corium.

*Amended, once the first real node was upgraded.* Staging streams. `cctl` opens
`POST /v1/upgrade/stage` and reads newline-delimited JSON until the pull ends: a
`progress` line for each line bootc prints, then one terminal record — `staged`
with what is now waiting, or `error` with what went wrong. It falls back to a
single JSON object and a status code only for the refusals that happen before
the pull starts, a malformed reference or an image the policy rejects, because
nothing has been written yet and a `400` or `403` is still the node's to send.

Two facts forced this, and both are worth recording because the first cut got
them wrong. A pull is hundreds of megabytes over whatever link a node has, so it
is minutes, not the "answers immediately or not at all" the client's timeout
assumed — and a 30-second client deadline did not merely cut a working upgrade
off, it *cancelled* it, because the abandoned request cancelled the context the
pull ran under, so the node's own 30-minute `pullTimeout` was never reachable.
And an operator watching a silent terminal for those minutes cannot tell a slow
pull from a stuck one. Streaming answers both: the node's `pullTimeout` is the
only clock, and the operator sees where the pull has got to. This is the same
`http.Flusher` and newline-delimited JSON the followed-journal route already
uses; it adds no dependency, which is the test this record sets for the daemon.

The same mismatch was latent in `drain` and `reset`, whose work is also minutes
— a drain waits on pod disruption budgets. They keep a single response, but the
client now gives them a deadline that matches the node's rather than the 30
seconds meant for the calls that answer at once. `apply` is untouched: it starts
a systemd unit and returns, and the drain and reboot happen in that unit, off
the request entirely.

**Node lifecycle**: reboot, shutdown, cordon, drain, and reset. These are
destructive and are marked as such in the schema.

Cordon and drain land differently from the rest, and the record should say so
rather than let somebody discover it: they work on controllers only. A node
acts on itself, and evicting a pod needs cluster admin credentials that only a
controller holds locally. Draining a worker stays a cluster operation, done
with `kubectl` against the kubeconfig this API will hand over — which is a
consequence of the node-local rule rather than a gap in it, and is the second
place that rule costs something visible. Reset is the strongest of
them: it takes the node out of the cluster, wipes `/var/lib/k0s`, and drops the
machine back to unenrolled — it is the one operation that returns a node to
maintenance mode, and it cannot leave it half-way.

### What it does not manage

No arbitrary command execution. There is no `exec`, no `shell`, and no endpoint
that takes a command line. The moment one exists this is SSH with a worse client
and a second authentication system to keep correct, and every argument for
keeping the surface small stops applying.

Almost no Kubernetes. The API does not proxy the apiserver, list pods, or keep
a kubeconfig on the operator's behalf. `kubectl` is not a gap.

The exception, added after the rest was built and working: it will hand over
the administrator kubeconfig k0s minted, once, on request. That was left out of
the first version of this record on the grounds above, and leaving it out was
wrong — it is the one Kubernetes-adjacent thing with no other answer. Without
it, getting a usable `kubectl` out of a freshly bootstrapped node means SSHing
into a controller and running `cat` on a file, which is the exact shape of
problem this API exists to remove.

It is `admin`, and it outranks everything else here: every other call acts on
one machine, and this one hands over a cluster. `reset` destroys a node; this
gives away every workload in the cluster, to somebody nothing here can take it
back from. The node says so in its journal for that reason.

What it does not become is a Kubernetes client. It reads the file k0s wrote,
rewrites the server address to one that will keep working — the virtual IP on
an HA control plane, since a kubeconfig aimed at one particular controller
stops working the first time that controller does — and returns it.

No package installation and no file writing.

#### Configuration, before a node is a node

The second exception, and it reverses something this record stated flatly:
**`cctl apply` writes a node's `corium:` document, and only before that node has
bootstrapped.**

What was written above — "this API cannot write a `corium:` block, and `cctl`
has no `apply-config`" — was aimed at the right target and drawn too wide. The
thing worth refusing is a configuration API for machines in service: a node
whose role, cluster or join token can be rewritten underneath a running
Kubernetes is a node whose configuration and behaviour are two different facts,
and decisions 6 and 7 exist to stop exactly that. That refusal stands. A
bootstrapped node returns `409` and is told to use `cctl reset`.

What the original wording also forbade, without meaning to, was the one case
maintenance mode was built for. A node held before bootstrap has no role, no
cluster and no workloads; there is nothing underneath it to be inconsistent
with. Yet it still had to be told what to be by cloud-init, which means the
bare-metal machine with no datasource — the case mode C names as its
justification — could be claimed and then had nothing to become. Enrolment
handed over ownership and left the node useless.

Three things make this narrower than it sounds:

- **It writes a source that already exists.** `/etc/corium/config.yaml` is
  first in decision 6's chain, ahead of cloud-init, and is there precisely for
  "bare metal, PXE and appliances [that] have no datasource". `cctl apply`
  fills a slot the design already reserved; it does not invent an authority.
- **It is bounded by the bootstrap, not by a permission.** The node refuses
  after it has bootstrapped, whoever asks and whatever role they hold. The
  marker in `/var` that stops a node re-bootstrapping is the same one that
  closes this door.
- **It does not reconcile.** The document is written once, by an operator, and
  read once, by the bootstrap. Nothing watches it, and nothing re-applies it.

`api.awaitConfig` completes it. Without it a claimed node bootstraps at once
with whatever it booted with, which for a machine nobody described is a
single-node cluster nobody asked for; with it the node holds for an operator to
say what it is, and only then builds anything. It is opt-in rather than
inferred from an empty document, because a node that waits for ever must do so
because somebody said so.

What the bootstrap waits on is a marker `corium-apid` writes in `/run`, not the
document. The two units are ordered against cloud-init and not against each
other, so an apply can land before the bootstrap has read anything — and a node
watching for a change that has already happened waits for ever. The marker
answers the question actually being asked, which is whether somebody answered.

The cost, stated plainly: a fleet can now be provisioned with an identical
four-line cloud-init that carries no secrets and says only `api.enabled: true`
and `api.awaitConfig: true`, with everything machine-specific arriving over the
API afterwards. That is a
genuinely different provisioning story from the one the rest of this document
describes, and it is closer to Talos's than anything else here. It is offered,
not preferred — modes A and B remain the unattended path, and a node that can
be described in cloud-init should be.

### Whether it runs at all

The API is opt-in, and the default is off:

```yaml
corium:
  api:
    enabled: true
```

A node with no `api:` block behaves exactly as a Corium node behaves today — no
daemon, no listening port, nothing to authenticate against, managed over the
console and SSH. Every existing configuration and every example in this
repository keeps working unchanged and gains no new attack surface by being
upgraded.

This is not only a compatibility convenience. A privileged daemon listening on
every machine in a fleet is a legitimate thing to refuse, and an operator who
refuses it should be able to say so in one line rather than by masking a unit
after the fact. `enabled: false`, written explicitly, masks
`corium-apid.service` at bootstrap so that nothing later re-enables it quietly.

Setting `operatorCA` or `operatorCAFrom` implies `enabled: true`, because
naming the CA that owns a node is an unambiguous statement that the node should
be manageable. Setting either of them *alongside* `enabled: false` is a
validation error rather than a precedence puzzle: the configuration says two
contradictory things, and the agent's contract is to fail before it mutates
anything.

### How trust is established

A node that runs the API needs exactly one thing to be manageable: the
certificate of the CA that signs operator client certificates. Note what that
is — a **certificate**, not a key. It is public material. It can be pasted into
a Git repository, a Terraform module, or an instance's metadata without leaking
anything, which is what makes the first of the three modes below defensible
rather than a compromise.

The private key of that CA lives on the operator's workstation or in whatever
mints short-lived client certificates for the team. It never reaches a node.

**Mode A — the certificate, in clear, in cloud-init.** The mode to reach for
when a PKI already exists.

```yaml
corium:
  role: worker
  api:
    operatorCA: |
      -----BEGIN CERTIFICATE-----
      ...
      -----END CERTIFICATE-----
```

Zero touch, no console, no post-boot step, and nothing secret in the metadata
service. Every node in a fleet carries the same block.

**Mode B — resolved at first boot from a `SecretSource`.** The same object
`join.tokenFrom` and `ha.authPassFrom` already use, with the same `waitFor`
semantics:

```yaml
corium:
  api:
    operatorCAFrom:
      url: https://pki.example.com/corium/operator-ca.pem
      waitFor: 15m
```

This exists because a CA is sometimes minted by the same run that builds the
cluster, and because an operator who would rather have metadata carry a pointer
than material should not have to argue for it. It reuses the machinery in
`internal/bootstrap/token.go` unchanged.

**Mode C — maintenance mode.** `enabled: true` and neither key set:

```yaml
corium:
  role: worker
  api:
    enabled: true
```

The node boots, reads its configuration, validates it — and stops before
bootstrapping k0s. `corium-apid` starts *unenrolled*: it serves TLS with a
self-signed certificate and answers exactly one RPC, `Enroll`. On the console,
on the serial port and in the journal it prints a pairing code and its own
certificate fingerprint:

```
Corium node is unenrolled and is not in a cluster.

  address       192.168.1.51:7443
  pairing code  K7QM-93XF
  fingerprint   SHA256:tQ2f...9c1a

  cctl enroll 192.168.1.51 --code K7QM-93XF
```

`cctl enroll` presents the code and the operator CA certificate. The node
compares the code in constant time, writes the certificate to
`/var/lib/corium/api/operator-ca.pem` with mode `0644`, marks itself enrolled,
restarts into the normal mTLS listener — and only then releases the bootstrap,
which proceeds from the cloud-init configuration it has been holding all along.

#### A node in maintenance mode is not in a cluster

That ordering is the load-bearing part, not an implementation detail.

The alternative — bootstrap first, enrol whenever somebody gets round to it —
produces a node that is running workloads, holds a kubelet credential and a
share of the cluster's secrets, and will obey the first stranger to reach an
unauthenticated port. It is the worst combination available: the machine is
valuable *and* it is unclaimed. Making the two states mutually exclusive
removes that combination from the design rather than defending it.

What an attacker can win by racing an unenrolled node is therefore a bare
machine that has joined nothing. That is still worth protecting — see below —
but it is a bounded loss, and it is a loss the operator finds out about,
because the node they were waiting on never appears in the cluster.

The same rule runs the other way and gives `reset` its meaning: a node cannot
go back to maintenance mode while it is a cluster member. Returning to
maintenance is a reset — the node leaves the cluster, `/var/lib/k0s` is wiped,
and it comes back up unenrolled and unbootstrapped, ready to be handed to
somebody else. There is no operation that leaves a machine both in a cluster
and unclaimed.

Enrolment carries a CA certificate. It may also carry a configuration —
`cctl enroll --config` — and when it does, the document is written before the
claim is recorded, so the node is never released to bootstrap with a
configuration its operator has already replaced. Sending one separately with
`cctl apply` does the same thing in two steps, and is what a node whose
bootstrap is still held or has already failed needs.

The node re-reads its configuration after the hold is lifted, which is what
makes either order work: the document the operator applied is the one that
bootstraps, on the same boot, rather than the one the machine happened to boot
with.

The cost is real and belongs in this record: **mode C is not zero touch.** A
three-node cluster in mode C is three consoles to visit before any node joins
anything, and `waitFor` on a join token does not help, because the joiner is
not waiting on the token — it is waiting on a human. Modes A and B remain fully
unattended, and an operator who wants both unattended provisioning and no
secret in the metadata already has mode A, since the value there is not a
secret.

The code is minted per boot from the kernel CSPRNG, uses an alphabet without
the characters people transcribe wrongly, and is worth around forty bits. It is
single-use, and five failed attempts end maintenance mode until the node is
rebooted — an operator who has fat-fingered a code five times will not mind
rebooting, and a script working through the keyspace gets one shot per boot.

Both the code and the count of wrong ones are kept in `/run`, which is what
makes "per boot" mean what it says. Holding them in the daemon's memory would
make them per *process*: systemd brings this daemon back after a crash, and a
restart would mint a second code while the first is still printed on the screen
above it, and hand a guesser their five attempts back. `/run` is a tmpfs, so
the state survives the restart, the reboot empties it, and none of it is ever
written to a disk — which matters, because the disk image of an unclaimed node
is exactly the one that gets passed around.

#### Why a pairing code rather than an open port

Talos is the obvious prior art and does not do this: its maintenance API on
`50000` accepts unauthenticated requests, `talosctl apply-config --insecure` is
the documented first step, and verifying the node's identity is opt-in through
`--cert-fingerprint`. That is a reasonable trade for Talos, where the prize for
winning the race is a machine that has no configuration yet.

A Corium node in mode C holds nothing either, which is the whole point of the
section above — so the argument has to be made on what happens next rather than
on what is there now.

An unenrolled node is not an empty machine. It is a machine holding a validated
configuration, a join token, and the intention to become a cluster member the
moment somebody claims it. Enrolling it is not reading its state; it is
acquiring the node for the rest of its life, since the CA it pins is the CA it
will obey until it is reset. Two things follow. An attacker who wins the race
gets a machine that proceeds to join the cluster with credentials the operator
supplied, under an attacker's CA. An attacker who merely wants to be a nuisance
wins by enrolling and doing nothing: the node is now claimed, the operator's own
enrolment fails, and recovering it means a physical reset.

Neither is catastrophic. Both are avoided entirely by a code that takes one
line of console output to read, and provisioning networks are shared often
enough that the window is not theoretical.

The code also closes the window in the other direction, which is the part worth
noticing: the code authenticates the operator to the node, and the fingerprint
printed beside it authenticates the node to the operator. An operator who skips
the fingerprint has not leaked anything — the CA certificate is public — but
has no way to tell a node that enrolled from an impostor that answered. Both
values arrive over the console, which is a channel an attacker on the network
does not have.

That last sentence is also the cost: **mode C needs console access** — serial,
IPMI, SOL, or the hypervisor's view. An operator with neither a console nor a
PKI is not served by mode C, and should use mode A.

#### And an opt-in that gives the code up

`api.insecure: true` drops the pairing code: the first client to reach an
unclaimed node claims it. That is Talos's model, argued against above, and it
is offered anyway — with the argument left standing rather than rewritten,
because it is still the reason this is not the default.

What makes it defensible where it is used is the rule the rest of this record
already enforces. An unclaimed node is in no cluster, so winning the race gets
a bare machine; and enrolment is still one-way, so the window closes the moment
anybody uses it. The cases it is for are real: a bench, a lab, a provisioning
network controlled end to end, a PXE fleet where visiting consoles is not a
thing anybody is going to do. Mode C otherwise asks for one console visit per
machine, which is a price some fleets will not pay and will route around with
something worse.

Three things keep it honest. It is refused anywhere it would silently do
nothing — with a CA configured, or with the API off — because a key that does
nothing is the mistake that costs a reboot cycle to find. The node says so on
its console, in words aimed at somebody who did not write the configuration
that opened it. And the node **records that its claim was unauthenticated**,
in `/var/lib/corium/api/claim.json`, which `cctl status` then shows: a node
holds the same pinned CA whichever way it was claimed, so without that record
there is no way to tell afterwards which of a fleet's machines were taken by
whoever reached them first.

#### The node's own certificate

The node mints its serving key at first boot and self-signs it. No private key
is ever carried in a configuration, which is the whole point of the scheme, and
it is why the operator CA cannot sign the serving certificate: the node has the
CA's certificate and not its key, and there is nothing online to ask.

`cctl` therefore pins a node's fingerprint the first time it talks to it and
records it next to the endpoint in `~/.corium/config.yaml`. In mode C the
fingerprint was on the console next to the code. In modes A and B it is in the
journal and in the instance console log, and `cctl` prints it and asks before
pinning unless `--fingerprint` was passed.

#### Precedence, and why enrolment is sticky

The whole resolution, in order:

| `api:` | Result |
|---|---|
| absent, or `enabled: false` | No daemon, no port. The node bootstraps as it does today |
| `operatorCA` set | Mode A |
| `operatorCAFrom` set | Mode B |
| `enabled: true`, neither set | Mode C: maintenance, and bootstrap is held |
| `enabled: false` with either key set | Validation error |
| both keys set | Validation error, matching `token` and `tokenFrom` |

Enrolment is recorded in `/var/lib/corium/api/`, and `/var` survives upgrades
and reboots. A node that has been enrolled — by any of the three modes — does
not return to maintenance mode when it reboots. This is not a detail: if it
did, anybody able to power-cycle a machine could take it, and the pairing code
would be protecting a door that reopens on its own. It is also implied by the
rule above, since a rebooting cluster member would otherwise come back up
unenrolled and still in the cluster.

Getting back to maintenance mode therefore means resetting the node, with
everything that implies. That is deliberately more than an operator who has
merely lost their CA key should have to do, so the recovery path is a different
one: `corium-agent api set-ca --file operator-ca.pem`, run as root on the node.
It is a local command, not a network operation — it opens no port and accepts
no unauthenticated RPC — and root on the console already owns the machine, so
it grants nothing that was not already granted. The cluster keeps running
throughout.

Replacing the operator CA on a *reachable* node is done over the authenticated
API, `cctl ca rotate`. Neither path goes back through maintenance mode.

### Authorisation

The client certificate's organisation carries the role: `corium:readonly`,
`corium:operator`, `corium:admin`. Read-only reaches state and journals;
operator adds services, upgrades, cordon and drain; admin adds reboot,
shutdown, reset and CA rotation. Three roles because two were not enough to
separate "let the on-call read the journal" from "let the on-call reboot it",
and four would be a policy language nobody asked for.

OIDC, raised in the issue, is deferred rather than rejected. It authenticates
humans far better than a certificate on a laptop does, and mapping its claims
onto these same three roles is a small change. What it cannot be is the floor:
it requires an identity provider to be reachable from the node, and the moment
this API is most valuable is the moment the cluster and the network around it
are least healthy. mTLS works on a machine that can reach nothing.

## Consequences

The daemon is privileged. Reading the journal, calling `bootc`, and rebooting
all require it, and no amount of capability trimming changes the fact that the
process can reboot the machine. That is the argument for the surface being as
small as it is, and the reason there is no `exec`. It also means SELinux policy
is part of the work, not a follow-up.

The daemon adds no third-party dependency. `net/http`, `crypto/tls` and
`encoding/json` cover the transport, so the API ships to every node without
enlarging what has to be trusted, audited or patched there. If that ever stops
being true, it is a reason to revisit this record rather than a detail to
absorb quietly.

A node that opts in grows a listening port; a node that does not, does not.
`7443` is clear of everything k0s binds — `6443`, `9443`, `8132`, `8133` — and
of the kubelet's `10250`. Whether it should be reachable from anywhere but a
management network is the operator's decision, and the documentation will say
so.

Mode C puts `corium-bootstrap.service` behind enrolment, which means the unit
ordering has to be got right rather than assumed. The service already carries a
comment about an ordering cycle that systemd resolved by silently deleting the
job, so the gate is a target the daemon reaches rather than another `After=`
edge on `cloud-final.service`. A node parked in maintenance mode must also read
as parked and not as failed: greenboot must not mark the deployment bad, the
unit must not sit in `failed`, and `systemctl list-units` should make the state
obvious to somebody who did not provision the machine.

Losing the operator CA key means losing the ability to manage every node that
trusts it, and the recovery is now a local one rather than a reset: console in,
`corium-agent api set-ca --file`, done. Nodes keep running Kubernetes
throughout, because none of this is in the data path. Losing the key is
therefore an inconvenience proportional to the number of consoles, not an
outage.

The `corium:` block gains one optional key with three spellings under it. `api:`
absent is the default and means no API at all, which keeps the promise that
every line of a Corium configuration is optional — and keeps this ADR from
changing the behaviour of a single node already in service.

## Alternatives considered

**SSH with a wrapper CLI.** Zero new attack surface, and it already works. The
reason it loses is authorisation: SSH grants a shell, and there is no way to
give somebody the journal without giving them the machine. Forced commands get
partway there and turn into a second, worse policy engine written in
`authorized_keys`.

**A Kubernetes-native design — a DaemonSet and CRDs.** Tempting, because the
cluster is right there and it would need no new transport. It fails on when it
is needed: a node whose kubelet is wedged, whose etcd has lost quorum, or which
is halfway through a bad upgrade is exactly the node an operator needs to reach,
and it is exactly the node a DaemonSet cannot answer from. Putting node
lifecycle inside the thing being upgraded also means the upgrade can sever the
connection that was driving it.

**A shared bearer token in cloud-init instead of mTLS.** Simpler, and it is
what several projects in this space do. It reintroduces the problem the
`SecretSource` machinery exists to avoid: a real secret sitting in instance
metadata, readable by anything on the node that can reach `169.254.169.254`,
identical across the fleet, and awkward to rotate. A CA certificate in the same
place leaks nothing.

**A cluster CA with its key in the configuration, as Talos does with
`machine.ca`.** It is a genuinely nicer end state — one anchor, every node's
serving certificate signed by it, no fingerprint pinning anywhere — and it is
ruled out by the same sentence: it puts a private key in the configuration. If
that key is in cloud-init, mode C has no reason to exist.

**Letting an unenrolled node join the cluster and enrol later.** This is what
the first draft of this record said, and it is worth keeping here because it is
the obvious design and it is wrong. It makes mode C fully unattended, which is
a genuine advantage, and it costs a node that is simultaneously valuable and
unclaimed — running workloads, holding cluster credentials, and waiting to obey
whoever reaches an unauthenticated port first. The pairing code was invented to
defend that state. Deleting the state is better than defending it.

**An always-on API with no way to switch it off.** Fewer states, one less key,
and a fleet where `cctl` always works. Rejected because a privileged daemon
listening on every machine is a reasonable thing to veto, and because a node
that has been running happily for a year should not acquire a new listening
port by being upgraded. Off by default also means this ADR can be implemented
without changing what any existing configuration does.

**gRPC instead of JSON over HTTP.** This is what the first version of this
record chose, on two real advantages: streaming is in the framework rather than
hand-rolled, and the client is generated from the schema so it cannot drift
from the server. Talos is the prior art and does exactly this.

It is reversed here on the cost, which is specific to what Corium is rather
than to APIs in general. `google.golang.org/grpc` brings `protobuf`, `x/net`,
`x/sys` and `x/text` behind it, and this repository has one dependency today.
Those modules would not land in an application — they would be baked into an
operating system image, on every node, in the read-only `/usr`, behind a
listener that runs as root. That is the one place in this project where
"prefer the standard library" is not a style preference, and the guidance in
`AGENTS.md` is explicit that every dependency here is attack surface that ships
to every node.

What is given up is smaller than it looks. `crypto/tls` does mutual TLS with
client certificate verification directly, and it is the same code path either
way. Streaming a journal is newline-delimited JSON and `http.Flusher`, which is
a few dozen lines rather than a framework. The generated client is the real
loss, and it is answered by the API surface being four endpoints rather than
forty, and by the client living in this repository next to the server.

`curl` working on a node that is half-broken turns out to be worth something
too, which had been filed as a consolation and is closer to a feature.

## What this does not settle

The wire schema, the endpoint paths, and the `cctl` command tree. Those are the
next pull requests. This record fixes the shape: off by default,
node-local, four surfaces, no exec, mTLS anchored in an operator CA, three ways
for that CA to arrive of which one requires nothing in cloud-init at all — and
the rule that holds the last of them together, that a node nobody has claimed
is a node in no cluster.
