CI: end-to-end validation of the FULL getting-started walkthrough on a REAL backend (incl. optional storage & LB)

Why (this must use a real backend)

An install-path-only test on kind with no backend is worth nothing — it would pass while every bug that actually bit shipped: hardcoded image ids (#13 (closed)), zero workers (#15 (closed)), the CSI driver-name mismatch (#21 (closed)), a never-binding PVC (#20 (closed)). All of those only surface against a real whitesky.cloud backend (real VMs, real disks, real provisioning). So this gate provisions the real thing and runs the documented procedure exactly as a new user would.

Scope — the COMPLETE getting-started section, end to end, real backend

Against a dedicated test location with real credentials, execute every page of docs/getting-started/ — including the optional steps — asserting each actually works, then tear it all down:

  1. prerequisites — required tools present; a fresh management cluster (kind-in-CI is fine for the management cluster).
  2. install-provider — clusterctl init --infrastructure vco:<tag> from the branch-built bundle → assert cert-manager + CAPI core/bootstrap/control-plane + the capvco manager all reach Ready; apply the CNI add-on CRS → assert the ClusterResourceSet exists. (Catches metadata dead-drops, cert-manager failures, a broken manager.)
  3. first-cluster — create the credential Secret; clusterctl generate cluster … --worker-machine-count 1 with real image ids; apply → real VMs provision; assert control-plane and worker reach Ready. (Catches #13 (closed) image ids, #15 (closed) worker count.)
  4. access-and-deploy — fetch the workload kubeconfig; assert both nodes Ready (CNI actually delivered via CRS); deploy the hello app, expose it in-cluster, curl it.
    • (optional) persistent volume — install the CSI add-on; create a PVC; mount it into the app; write then read back; confirm it survives a pod restart. (Exactly what would have caught #21 (closed) — a PVC that silently never binds — and the #20 (closed) gap.)
    • (optional) LoadBalancer — enable the service-lb add-on; a Service type=LoadBalancer gets a real external IP; reach it.
  5. teardown — delete the cluster; assert all whitesky.cloud resources (cloudspace / VMs / disks / LB / DNS) are released — no leaks.

Rule: if it's in the getting-started walkthrough — including every "(optional)" block — it's tested here. A green run means a new user can follow the docs verbatim and succeed.

Hard requirement — drive the DOCUMENTED commands verbatim, with NO out-of-band fix-ups. The harness executes the exact commands from docs/getting-started/ and the guides they link (e.g. storage, load-balancing) as written. It must NOT apply an add-on CRS, create a Secret, set a value, or run any step the docs don't explicitly tell the reader to run. If a documented step is missing, wrong, or out of order, the run fails there — a docs gap is a first-class failure, exactly like a code bug. Rationale: #23 (closed) (LoadBalancer) had both — a code bug (vco-lb-credential never injected) and an undocumented service-lb CRS-apply step — and only a verbatim run catches both; a test author who "knows" to apply the CRS themselves would turn the run green while the docs stay broken, defeating the entire purpose. The only things allowed out-of-band are the inputs the docs legitimately require the reader to supply (credentials, image ids, location, portal host), passed as documented inputs — anything the docs omit is a bug the run must expose, never paper over.

Requirements / implications (a heavy, credentialed gate — by necessity)

  • Real credentials (masked/protected CI variables), a dedicated test customer + location; per-cluster credential model (§12/§18).
  • Docker/DinD runner for the kind management cluster; the workload cluster provisions on the real backend via the provider.
  • Guaranteed teardown even on failure (trap + a reaper for stragglers) — no orphaned cloudspaces/VMs/disks/DNS, no cost leaks. Unique names per run.
  • Time/cost: provisions real infra (several minutes). It needs secrets, so it can't run on fork MRs. Placement: a required gate on MRs to protected/internal branches (and/or pre-merge / pre-release). The original "cheap, no-secrets, every-MR" framing is dropped — it would have caught none of the real bugs.

Relationship to #18 (closed)

#17 (closed) = the documented getting-started happy path (base cluster + app + optional storage + optional LB), proven end-to-end on a real backend, and the reusable harness. #18 (closed) = the broader feature matrix (geo single/multi-location, HA, CCM, backup, air-gapped, disks, upgrades, encrypted storage) built on that harness.

Acceptance

Each run executes every getting-started page — including the optional storage and LoadBalancer sections — against a real backend, and any breakage (install, provisioning, CNI, worker join, PVC bind+mount+read/write, LB IP allocation, teardown/leak) fails the pipeline with a clear pointer to the failing step.

Related: #13 (closed)/#15 (closed) (would be caught at provisioning), #20 (closed)/#21 (closed)/#22 (the storage steps this now exercises), #16 (closed) (release cost/reliability).

Edited by Geert Audenaert