Neon Law | Workshops | Deploy The Navigator | Step

Ship and verify

Provisioning gives you an empty cluster; now pin one deployment to one published release. The --deployment flag selects the private row that supplies its project, cluster context, namespace, image name, hosts, buckets, SQL instance, required browser OAuth client, and runtime Secret name. First-install order is load-bearing: apply the deployment's Secret Manager objects, install observability so navigator-otel-env exists, render the release, then apply it:

Before shipping from a checkout whose deployment changes have not reached your installed binary, install that checkout's CLI. A stale global binary may enforce an obsolete ship contract even when the selected deployment's config correctly carries the current production profile:


cargo install --path cli --force

An operator wrapper must make the same guarantee before it changes Kubernetes or GCP: build the selected checkout's cli package, then prove that navigator ops secrets apply --help exists. Do not fall back to an arbitrary pre-existing target/release/navigator; a stale binary can lack a subcommand the current runbook requires and stop only after it has already refreshed cluster credentials. Browser OAuth values belong in the deployment's tree before the wrapper starts, so preparation is non-interactive. Keep explicit confirmations only for irreversible resource retirement and the live release roll.

The failed guard runs before manifest rendering or cluster mutation. Install the matching CLI, set both environment and credential profiles to production in the deployment's config.toml, then run the complete sequence:


navigator ops secrets apply --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR"

navigator ops observability --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR"

navigator ops ship --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR" --tag YY.M.D --dry-run

navigator ops ship --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR" --tag YY.M.D \
  --image-digests "$IMAGE_DIGESTS"

ops observability is safe before the application Deployments exist: it creates the namespace-scoped collector and navigator-otel-env ConfigMap, then skips the optional Deployment patch because ops ship renders that wiring into new Deployments. A cold Autopilot cluster may take several minutes to create its first nodes and start the managed Prometheus admission webhook. The CLI retries an idempotent IAM binding while a new navigator-otel Google service account propagates. Before it applies collector-monitoring.yaml, the Rust CLI uses Google ADC and the Container API to read the selected GKE cluster endpoint and CA, then queries the managed gmp-operator Endpoints object with its Kubernetes client. It waits only while that object has no ready addresses; a RUNNING GKE cluster is not treated as proof that the admission webhook is ready. The endpoint wait is bounded to three minutes and prints each attempt. If it expires, inspect the managed operator rather than deleting the collector or patching the Secret:


kubectl --context "$NAVIGATOR_GKE_CONTEXT" -n gke-gmp-system \
  get deployment,pods,endpoints gmp-operator

When the managed operator becomes ready, rerun the exact same navigator ops observability or three-deployment operator command. Namespace creation, Secret reconciliation, Google service-account creation, IAM bindings, and collector manifests are all idempotent; a partially completed first run is a resume point, not a cleanup instruction.

The staging dogfood run also proved two quota-independent defaults. Autopilot clusters are created with --enable-private-nodes, so a new region does not need one public in-use address per node. The link-out compatibility writer uses the standard persistent-disk class, not an SSD-backed class; it must not consume SSD_TOTAL_GB for an otherwise empty mount.

Presenter notes

GitHub Actions publishes neon-server and navigator-workflows-service to $NAVIGATOR_IMAGE_REGISTRY_HOST/navigator. The repository is private, so a deployment's node identity needs roles/artifactregistry.reader, and images are deleted 7 days after push. ops ship refuses latest, verifies every selected image exists, renders the embedded manifests with no unresolved placeholders, diffs before applying, preflights the runtime Secret, waits for all rollouts, and registers the worker. The --dry-run form prints an offline local plan. It does not verify cloud authentication, image availability, signatures, Secret Manager state, or live cluster drift. For an actual roll, follow docs/manual-roll.md and supply --image-digests from the matching signed release map.

Then confirm the service is live — and you can do it from the page itself. The site footer renders the deployed release as "Neon Law Navigator YY.M.D", so the moment your new image is serving traffic the footer changes: that is your end-to-end "it worked." For a scripted check:


curl -fsS https://www.your-domain.example/readyz
curl -fsS https://www.your-domain.example/version   # {"release":"YY.M.D","commit":"…",…}

web exposes a readiness endpoint that returns 200 OK only once it has a database connection and its dependencies in hand, and a /version endpoint whose release field is the very same YY.M.D the footer shows. A 200 on /readyz means the same stack our firm runs is now answering on your own cloud, and the release field tells you which dated image landed without shelling into a pod.

To verify a configured row after a release, load the row's private config and check the public host it provides. Repeat for every configured row after staging is verified:


: "${NAVIGATOR_DEPLOYMENTS_DIR:?set this to the operator-managed private deployment root}"
: "${ROW:?set this to the selected row name}"
TAG="${TAG:?set this to the shipped release tag}"
CONFIG="$NAVIGATOR_DEPLOYMENTS_DIR/deployments/$ROW/config.toml"
host=$(sed -n 's/^NAVIGATOR_PUBLIC_HOST = "\(.*\)"$/\1/p' "$CONFIG")
curl --fail --show-error --silent "https://${host}/readyz" >/dev/null
curl --fail --show-error --silent "https://${host}/version" \
  | jq --exit-status --arg tag "$TAG" '.release == $tag' >/dev/null
echo "OK ${host} ${TAG}"

A successful apply alone is not release evidence. Record the selected row, tag, verified endpoints, and outstanding checks in the approved deployment record. Keep hostnames and other operational coordinates in the private deployment system.

Set exactly one website to one published version:

Choose ROW from the operator-managed deployment root (host is that config's NAVIGATOR_PUBLIC_HOST), set TAG, then run:


ROW="${ROW:?set this to the selected row name}" TAG="${TAG:?set this to the release tag}"
CONFIG="$NAVIGATOR_DEPLOYMENTS_DIR/deployments/$ROW/config.toml"
host=$(sed -n 's/^NAVIGATOR_PUBLIC_HOST = "\(.*\)"$/\1/p' "$CONFIG")
gcloud auth login --force --update-adc

set -a; eval "$(grep ' = "' "$CONFIG" | sed 's/ = /=/')"; set +a
gcloud container clusters get-credentials "$NAVIGATOR_GKE_CLUSTER_NAME" \
  --region "$NAVIGATOR_GCP_LOCATION" \
  --project "$NAVIGATOR_GCP_PROJECT_ID"

navigator ops secrets apply --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR"
navigator ops observability --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR"
navigator ops ship --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR" --tag "$TAG" --dry-run
navigator ops ship --deployment "$ROW" --deployments-dir "$NAVIGATOR_DEPLOYMENTS_DIR" --tag "$TAG" \
  --image-digests "$IMAGE_DIGESTS"

curl --fail --show-error --silent "https://${host}/readyz" >/dev/null
curl --fail --show-error --silent "https://${host}/version" |
  jq --exit-status --arg tag "$TAG" '.release == $tag'

Change only ROW and TAG; the host follows from the private config. The source tree carries no row-to-host map. After the selected row's DNS and certificate are ready, its host should answer /readyz and /version directly.

Roll and verify staging before production. A successful Kubernetes apply is not completion: /readyz must succeed, /version.release must equal tag, and the website must open in a browser.

Do not present the Slack message as a deployment command. It is the evidence handoff after every deployment's health, version, and browser checks pass. Demonstrate the single-site recipe separately: one deployment, one immutable tag, one cluster context, and one host verification. Keeping those values paired is what prevents an operator from rolling the wrong site.

View all slides Open display