Agent Control Plane
This page is my own implementation of an agent control plane: a service I built and run that watches real deploys of another one of my projects, has an AI model read the actual code change behind each one, writes and runs browser tests against it, and publishes the result on a public dashboard. It's a working demonstration of AI-assisted software engineering, not a description of the concept.
The general pattern: this is how an agent control plane fits into a delivery pipeline, not a diagram of one specific deployment.
Here's the real implementation.
Two cooperating services, both live right now: a small REST API that ships on every push, and a control plane that watches those deploys, reads the actual code change behind each one, writes and runs browser tests against it, and publishes the full result on a public dashboard, generated spec included. Everything below this point is real, not conceptual: every run on that dashboard reflects an actual push that happened, went through the pipeline described here, and reported in.
Real deploys, real tests
Every push becomes a tracked run with its own status and history, not a seeded demo dataset.
Diff-aware test generation
Specs are written against this deploy's actual diff and schema, not a static suite hand-written once.
Verified at the boundary
HMAC-signed webhooks, isolated databases, credentials that never touch version control.
Live Overview
api-test-gateway View source →
A Fastify/TypeScript REST API for a small project and task management domain (“TaskBoard”), backed by Postgres. It's deliberately compact but exercises real-world surface area: authentication, per-project ownership, role-based authorization, and status transition rules enforced server-side, not just in the client.
On every deploy it sends a signed webhook announcing what shipped: commit, branch, and environment. It also carries a runtime error reporter that can notify the control plane the moment something breaks in production, with sensitive data stripped before it ever leaves the process.
dev-agent-control-plane View source →
Receives those signals, verifies them, and turns each one into a tracked run: a record with its own status, timeline, and history, visible on the public dashboard within seconds of a real deploy landing. It's built as a small modular service with separate processes for the HTTP surface and the background work, so a slow job can never block an incoming webhook.
Every webhook is authenticated with an HMAC signature bound to a timestamp, checked against the exact bytes sent, so a request can't be replayed or reconstructed after the fact. Duplicate deliveries are detected and de-duplicated at the database level, not by trusting the sender not to retry.
Live Pipeline
From a commit to a set of passing browser tests
The webhook doesn't just log that a deploy happened, it kicks off a pipeline that reads the diff, decides what could have behaviourally changed, writes Playwright tests for exactly that change, runs them against the freshly deployed instance, and tries to repair its own mistakes before giving up. Every stage below is its own recorded step on the run's timeline, so you can open any run on the dashboard and see the diff, the change summary, the test plan, the generated spec, and the execution result, in order, not just a final pass/fail.
This is what makes the run detail page worth opening instead of just a green check mark: the spec you see there was written against this deploy's actual diff and this API's real request/response shapes, not a static suite that was hand-written once and never revisited.
Two things keep this catching real problems instead of just producing noise. First, it never gets to guess the API's shape: the contract pack above is built straight from the checked-out route handlers and schemas, so a generated test expects the response the endpoint actually returns, not a plausible-sounding one. A failure is far more likely to mean the deploy broke something than that the test was wrong. Second, execution happens against the instance that was just deployed, over the same network path a real client would use, so a failing test is catching an issue in the code that's live right now, at the moment it went live, not in some separate environment days later.
A few of the choices above are deliberate, not defaults left switched on. Retrying is capped at five attempts and only fires when a failure looks like a mistake in the generated test itself; anything that looks like an environment problem or a genuine app regression stops immediately and gets reported as-is, so a bad run can't quietly loop forever chasing a bug that was never in the test to begin with. Validation runs before execution as its own, cheaper step, a plain check that the generated source is well-formed and safe to run, so a malformed spec never reaches a real browser session against staging. And every accepted spec gets its own row tied to its run rather than overwriting one file per project: the last five feed back in as prior coverage when planning the next run, but nothing here silently deletes or rewrites history, each run's result stays exactly as it was reported at the time.
Getting api-test-gateway onto the internet
Each push to main runs through a small pipeline:
- Lint, typecheck, and a full test suite, including an integration matrix covering every endpoint against every role and business rule
- A container image built and published, tagged to the exact commit
- A deploy step that connects over SSH with a dedicated, narrowly-scoped key, pulls the new image, runs database migrations, and waits for a health check before calling it done
- If any step fails, the previous version keeps running. Nothing goes live half-deployed
Live Architecture
The gateway and the control plane are deployed as separate services with one narrow connection between them: signed HTTP, nothing else. Neither reaches into the other's database or process.
Keeping the control plane's data boundaries tight
The control plane never touches the gateway's database directly: the only connection between them is signed HTTP. Its own database, admin API, and background worker are kept off the public internet entirely; only the reverse proxy in front of it is reachable from outside, and it in turn only exposes the dashboard and a read-only public API. Every credential (database passwords, webhook secrets, admin tokens) lives in an environment file that never gets committed, generated fresh per environment rather than reused.
Why the API reference asks for a password
The API reference link is real, live Swagger UI generated straight from the gateway's own request/response schemas, not a static mockup. It sits behind HTTP Basic Auth: browsing it means asking me for the credentials. The endpoints it documents each require their own bearer token on top of that, so the docs page being gated just keeps the API's shape itself from being crawled by anyone passing by, while the endpoints stay independently protected either way.
Self-registration is also switched off on this deployment, so the only way to get a bearer token to try the protected endpoints is to ask me for one.
Live Integrations
The pieces this system is actually built from, not a generic integrations grid:
- GitHub Actions, running lint, typecheck, and the full test suite on every push before anything gets built
- Playwright, driving the real headless-browser runs against the freshly deployed instance
- Postgres, backing both the gateway's application data and the control plane's run history
- A swappable model layer for the AI steps: this deployment runs on a hosted LLM provider by default, but the same structured calls work unchanged against a different hosted provider or a locally hosted model, swapped with environment variables and no workflow rewrite. Everything around those model calls, the diff reading, the contract pack, validation, execution, repair policy, persistence, and dashboard, stays deterministic
Background: what an agent control plane is, and the problem it solves. The live implementation above is what this looks like built.
What is an Agent Control Plane?
A central platform for creating, deploying, managing and monitoring AI agents, so every team building on it shares one set of services (auth, orchestration, scheduling, governance, monitoring) instead of each reinventing its own. The full breakdown is in Capabilities below.
The Problem This Is Solving
Without a shared platform, every team ends up solving the same problems differently, and usually solving them worse the second and third time.
Architecture
A general reference architecture for this category of platform, not a diagram of what's deployed below. See the "Live" architecture panel further down for the actual deployed system.
Agent Lifecycle
How an agent typically moves from an idea to something running in production, and staying observable once it's there.
Create
A developer defines the agent's task and boundaries.
Commit
The definition lands in version control like any other code.
CI/CD Validation
Automated checks confirm the agent behaves as specified.
Security Checks
Policy and permission review before anything reaches production.
Deploy
The agent becomes available for the control plane to run.
Run
The agent executes against real work.
Observe
Every run is logged and attributable.
Optimise
Cost, latency, and accuracy get tuned from real usage data.
Version
Changes ship as a new version, not an untracked overwrite.
Capabilities
The services a control plane typically provides, grouped by what they're actually for.
Orchestration & Runtime
Orchestration
Coordinates multi-step agent work across a workflow.
Scheduling
Runs agents on triggers, timers, or events.
Memory
Durable context carried across runs.
Governance & Security
Authentication
One shared identity layer for agents and callers.
Governance
Central policy over what agents can do.
Approval Workflow
Sensitive actions can require human sign-off.
Observability & Cost
Monitoring
Every agent run visible in one place.
Auditing
A reviewable record of what ran and why.
Cost Tracking
Spend attributed back to its source.
Integration
Model Routing
Selects which model handles a given request.
Integrations
A shared layer for connecting agents to external systems.
What building this actually involved
Two deployed services, one signed integration between them, and a pipeline that reads a real diff and turns it into a real, executed browser test. Underneath the AI steps, everything else, the webhook verification, the retry and repair policy, the database boundaries, the CI/CD pipeline, is deterministic code I wrote and can explain line by line.
Tech stack