Overview
This week the new LegionEdge consoles went live at cloud.legionedge.ai: the cloud console, the IAM app where you sign in and run your organization’s identity, and the admin panel our platform team works from. This post covers how they’re put together and how a commit on main reaches production.
Two constraints shaped the design. Three apps had to share one session, and a deploy couldn’t depend on anyone remembering a manual step.
One origin, because of one cookie
Better IAM’s session cookie carries the __Host- prefix. Browsers only accept such a cookie when it is Secure, has Path=/ and sets no Domain, which ties it to a single host. If the cloud console, the IAM app and the admin panel each had their own subdomain, signing in to one would leave you signed out of the other two.
So production is a single origin built from Next.js multi-zones. apps/cloud owns / and every /<your-alias> path. The IAM app is mounted at /iam and the admin app at /admin. Only the cloud app sits behind the load balancer. Its rewrites forward /iam and /admin (pages, API routes and the /_next assets under each zone’s base path) to the other two services inside the cluster. They run as beforeFiles rewrites, so those paths never reach the cloud app’s own organization route.
Sharing one origin also explains why some organization aliases are reserved: iam, admin, api, _next, legionedge, login and signup would collide with top-level paths. Because client components link across zones, the origin is a build argument, and each image set is built for exactly one origin.
The short hostnames are redirects, not separate apps. iam.legionedge.ai, admin.legionedge.ai and app.legionedge.ai forward into cloud.legionedge.ai and keep the path, so iam.legionedge.ai/login lands on cloud.legionedge.ai/iam/login. Each hostname has its own Certificate Manager certificate, so adding one never touches the certificate that serves live traffic.
The edge
Traffic arrives at a GKE Gateway, a global external Application Load Balancer on a static IP, with TLS 1.2 or later and a redirect from HTTP to HTTPS. Certificates are authorized through DNS, so they issue before a domain points at the load balancer and a cutover has no gap without a certificate.
Cloud Armor sits in front of the Gateway. It runs the OWASP Core Rule Set 3.3 at its lowest-false-positive sensitivity, along with adaptive DDoS protection, a per-IP rate limit and a tighter budget on the sign-in and sign-up paths. One lesson from launch day: CRS rule 921150, which looks for CR/LF in argument names, matched every Next.js server-action form post, sign-in included. We opted out of that one signature and kept the rest of the protocol-attack rules.
From push to rollout
Every push to main starts the Deploy workflow. No credentials are stored in it. GitHub’s OIDC token is exchanged through Workload Identity Federation for a deployer service account that only accepts this repository’s main branch, and no service account keys exist.
The build stage is a matrix of six images: cloud, iam, admin, website, worker and compute. Each builds in its own job with docker buildx bake, using a GitHub Actions cache scoped to that app. Registry tags are immutable, so every tag combines the commit, the run number and the attempt, and a re-run gets a tag of its own. Bake pushes an SBOM and SLSA provenance with each image.
Only one deploy runs at a time. A newer push waits for the current rollout to finish instead of cancelling it. The deploy script reads the build metadata and pins every image in the Kustomize overlay by digest. The tag is only a label, and Kubernetes runs exactly the bytes that were built.
Migrations before code
The rollout happens in three phases. First, configuration goes out: ConfigMaps, Secret Manager provider classes, NetworkPolicies and service accounts. Second, a one-off Kubernetes Job runs pnpm db:migrate on the new iam image. That applies every schema’s pending SQL files from packages/*/migrations/<schema>/ in file-name order, each under its own Postgres advisory lock, and records them in that schema’s migrations table. Third, if and only if the Job succeeded, everything else is applied and the script waits on each deployment’s rollout.
When a migration fails, nothing rolls out and the running pods stay as they were. We added this ordering after launch. Our packages also migrate on first use, but only on code paths that open their schema. On the fresh production database, the admin Plans page queried a table that no code path had created yet. Running every schema’s migrations ahead of the rollout removes that whole class of bug, and new code never runs against an old schema.
What runs in the cluster
Images are built in a Debian slim stage and run on distroless Node 24 as a non-root user. They have no shell and no package manager, and base images are pinned by digest. Binary Authorization admits only images from our own registry plus the pinned Cloud SQL Auth Proxy. Pods run under the restricted Pod Security standard with a read-only root filesystem, every Linux capability dropped and no service account token mounted.
The network denies everything by default and then allows only the paths the platform uses. Nodes are private, and Cloud SQL has only a private IP. Every database connection goes through the Cloud SQL Auth Proxy over mTLS. Secrets live in Secret Manager and reach pods as files through the GKE CSI driver, and a small preload script turns those files into environment variables. No Kubernetes Secret holds a credential.
Background work starts with a CronJob that calls the IAM app’s cron route once a minute, authorized with a shared secret. Each tick delivers Better IAM’s outbox (every invitation, password reset and sign-in code, sent through Mailjet) and dispatches its events and sweeps. A Postgres ledger runs each job once per tick, however many IAM pods there are. Next we’re moving the platform’s own billing, metering and signup jobs onto the same scheduler.
