Ship and verify
Provisioning gives you an empty cluster; now pin one deployment to one published release. The --deployment flag
selects the deployments/<name>/config.toml that supplies the exact project, cluster context, namespace, image name,
hosts, buckets, SQL instance, required browser OAuth client, optional post-registration Gemini client, and runtime
Secret name. First-install order is load-bearing: apply the deployment's Secret Manager objects, install observability
so navigator-otel-env exists, render the release, then apply it:
Before shipping from a checkout whose deployment changes have not reached your installed binary, install that checkout's
CLI. A stale global binary may enforce an obsolete ship contract even when the selected deployment's config correctly
carries the current production profile:
cargo install --path cli --force
An operator wrapper must make the same guarantee before it changes Kubernetes or GCP: build the selected checkout's
cli package, then prove that navigator ops secrets apply --help exists. Do not fall back to an arbitrary
pre-existing target/release/navigator; a stale binary can lack a subcommand the current runbook requires and stop only
after it has already refreshed cluster credentials. Browser OAuth values belong in the deployment's tree before the
wrapper starts, so preparation is non-interactive. Keep explicit confirmations only for irreversible resource retirement
and the live release roll.
The failed guard runs before manifest rendering or cluster mutation. Install the matching CLI, set both environment and
credential profiles to production in the deployment's config.toml, then run the complete sequence:
navigator ops secrets apply --deployment <row> --deployments-dir .
navigator ops observability --deployment <row> --deployments-dir .
navigator ops ship --deployment <row> --deployments-dir . --tag YY.M.D --dry-run
navigator ops ship --deployment <row> --deployments-dir . --tag YY.M.D
ops observability is safe before the application Deployments exist: it creates the namespace-scoped collector and
navigator-otel-env ConfigMap, then skips the optional Deployment patch because ops ship renders that wiring into new
Deployments. A cold Autopilot cluster may take several minutes to create its first nodes and start the managed
Prometheus admission webhook. The CLI retries an idempotent IAM binding while a new navigator-otel Google service
account propagates. Before it applies collector-monitoring.yaml, the Rust CLI uses Google ADC and the Container API to
read the selected GKE cluster endpoint and CA, then queries the managed gmp-operator Endpoints object with its
Kubernetes client. It waits only while that object has no ready addresses; a RUNNING GKE cluster is not treated as
proof that the admission webhook is ready. The endpoint wait is bounded to three minutes and prints each attempt. If it
expires, inspect the managed operator rather than deleting the collector or patching the Secret:
kubectl --context "$NAVIGATOR_GKE_CONTEXT" -n gke-gmp-system \
get deployment,pods,endpoints gmp-operator
When the managed operator becomes ready, rerun the exact same navigator ops observability or three-deployment operator
command. Namespace creation, Secret reconciliation, Google service-account creation, IAM bindings, and collector
manifests are all idempotent; a partially completed first run is a resume point, not a cleanup instruction.
The staging dogfood run also proved two quota-independent defaults. Autopilot clusters are created with
--enable-private-nodes, so a new region does not need one public in-use address per node. The link-out compatibility
writer uses the standard persistent-disk class, not an SSD-backed class; it must not consume SSD_TOTAL_GB for an
otherwise empty mount.