Case study · Live at gatewayinfra.ai

Gateway.

Draw a cloud architecture on a canvas, or hand Gateway a team's GitHub repo, and it turns that into a running distributed system, with real Terraform and real Helm underneath. It's live, and it's a company I've built.

Services

AI agents

10 min

Env setup

−40%

LLM cost

The problem

Designing a system and shipping it are two different jobs

You draw an architecture in Lucid or Excalidraw, then translate it by hand into infrastructure code, service scaffolds, and a build pipeline. Every one of those hand-offs is a chance to drop a dependency, forget a firewall rule, or quietly pick a different region than the drawing implied. The picture and the running system start drifting apart the day the first ticket closes.

You also get no feedback while you're designing. Nobody tells you the diagram has a single point of failure or an expensive mistake in it until code review, or staging, or the meeting after the outage. Gateway is one artifact that carries a project from source code to a running, monitored system, so the design and the thing in production stay the same object.

How it works

Six steps, start to finish

Ingest

Hand Gateway a team's GitHub repo, or draw the system on the canvas. It reads what's there and works out what the system actually is.

Design

The architecture lands on a canvas where every box is a real cloud service, not a shape. The same design exports as deployable infrastructure code.

Simulate

Load, failure, and security runs against the design, so you find the bottleneck and the single point of failure before you ship it.

Test

Gateway writes and runs its own tests against the running system, including ones that break things on purpose. Merge a PR and a GitHub webhook stands up a live end-to-end environment that lasts 30 minutes, so every change gets tested before it's promoted.

Deploy

It provisions the real thing in the cloud and hands back a live URL. Each team gets dev, staging, and production environments as generated Terraform and Helm charts, can promote between them, and can add more stages when needed. Setup that took 2 days takes 10 minutes.

Observe

Metrics, logs, and alerts come back as a bundle, opened as a pull request you can read and merge.

The hardest thing

Letting an AI build real infrastructure without letting it do damage

Gateway lets an AI design your system and then actually builds it in a real cloud account. That means a model is producing something that runs with real credentials against real money. The hard part was never getting the AI to write something plausible. It was making sure that what came out of it could not do harm.

There are four things standing between the model and your account.

The model never writes deployment code directly. It fills in a fixed set of approved building blocks, and our own code turns that into the real configuration. The model picks from a menu instead of writing whatever it likes.
Before anything runs, a separate check reads every generated file and rejects anything that could execute its own commands. The generator can't produce those in the first place, but a third-party module someone imports could, so the check runs every time regardless.
Each build runs in a throwaway workspace with its cloud access cut down to just the job in front of it. If a build goes wrong, it can't reach anything outside itself.
Text that comes from a user's canvas or repo gets stripped of hidden characters and walled off before the model reads it, so a comment buried in someone's code can't quietly rewrite the agent's instructions.

The bug I'm glad I caught

One of those checks is a scanner that blocks any deployment with a high-severity problem in it. It had a hole. Whenever the scanner itself failed, because the tool was missing or the scan timed out, it reported a pass.

So in any environment where that tool was not installed, the check quietly did nothing while still looking green. The gate was there, it just was not holding anything back. I rewrote it to fail closed, so a scan that cannot run counts as a scan that failed, and added a startup check that refuses to boot the service at all if the scanner is missing.

Environments

Every deploy is append-only

Each deployment lives in an environment with a full history. Promoting or rolling back is not a change to what's already there. It's a new revision applied on top. Nothing gets destroyed, so you can always see exactly what was running and when, and you get back to an old state by moving forward rather than undoing. Container images and databases are pinned per revision, so a rollback brings back the whole set, not just the code.

The AI layer

Five agents, one job each

When you ask an agent about a component, it gets that piece of your design plus everything connected to it, put together fresh for that one request. That context comes from RAG over your infrastructure, and each agent's steps run as a LangGraph graph. It's answering about what's on your screen, not about system design in general.

Guide

Walks you through designing a system, asking about scale and constraints as you go.

Tutor

Explains why a component is there, what it does, and the trade-offs behind it.

Reviewer

Audits a finished design and scores it on scalability, reliability, maintainability, performance, and security.

Generator

Turns a plain-language problem statement into a first-draft architecture on the canvas.

Atlas

Diagnoses deployments. When a build fails and the usual rules don't explain it, Atlas reads the logs and tells you why.

Architecture

How it's built

The services

25 FastAPI microservices sit behind one API gateway that handles login and rate limiting. They talk over RabbitMQ, which carries the asynchronous work and soaks up spike loads, and share a common library.

Where it runs

Gateway runs on AWS EKS and EC2, with Helm charts per environment and Terraform managing the infrastructure underneath. Each team's dev, staging, and production environments are generated the same way, with promotion between stages. When a user deploys their own system, Gateway stands that up separately on ECS, with a load balancer and a managed database, provisioned with Terraform and built with CodeBuild.

The AI layer

Every AI feature, from code-to-design to triage, runs as LangGraph agent steps with RAG supplying the infrastructure context. Bedrock then picks the best model for each task, which cut LLM cost by 40%. Each agent has its own prompt and its own job, and none of them share a brain, so a bad answer stays in one place.

State and monitoring

Postgres and Redis hold state. Logs and metrics are structured so they can export to CloudWatch, Prometheus, or Datadog without rewriting anything.

What I'd do differently

I'd use Gateway to plan

Gateway is good at laying a system out and letting you test the shape of it before you commit to building it. That is exactly what I needed for the last several months, and I never once pointed it at itself.

There is some irony in building a planning tool and then not planning with it. If I started over I would map the whole thing on the canvas first, run the failure simulations against my own design, and find the awkward service boundaries there instead of finding them three months in with code already written against them.

FastAPIAWS EKSEC2TerraformHelmRabbitMQAmazon BedrockLangGraphRAGReactPostgresRedisCodeBuild