case study / Personal project · 2026

OrchestrAI: AI agents with a human approval gate

A solo-built, local multi-agent system for the software lifecycle: six specialised agents, an LLM supervisor, and no file write or command without human approval. Finalist at DevOpsDays Cairo 2026.

Client
Personal project
Period
2026
Role
Solo builder
Result
DevOpsDays Cairo finalist
Stack
TypeScript, Bun, Hono, MCP, LangGraph, AG-UI, Docker, GitHub Actions
requestOrchestratorLLM routerLangGraph supervisorDevOpsTestingDocsSecurityReviewCodertools shared over MCPhuman approval before every write

In short

  • Built OrchestrAI on my own: a local system where an orchestrator hands software tasks to six specialised AI agents for DevOps, testing, documentation, security, code review and coding.
  • Every action that writes a file or runs a command stops for human approval, in a browser dashboard or a terminal UI.
  • Built spec-first: more than 140 numbered specifications and about 1,600 automated tests, all checked in CI.
  • Ships as one binary for Linux, macOS and Windows, or runs with npx orchestrai.
  • Finalist at DevOpsDays Cairo 2026, in the top 20 of 142 submissions.

Context

AI agents can write a Dockerfile, a CI pipeline or a test suite in seconds. Most DevOps engineers still would not let one change a repository or run a command unsupervised, and they are right. OrchestrAI started as my DevOpsDays Cairo 2026 entry to test one idea: several agents can share the work across the software lifecycle, as long as a human keeps the final say on every change.

Constraints

  • Local first: it runs against a real project folder on the developer’s machine, with their own LLM provider key.
  • Fail closed: a wrong file write or command is worse than no answer, so every uncertain path has to stop, not guess.
  • Time: about seven weeks from first commit to the final demo, built alone. A reliable end-to-end demo mattered more than production scale.

Decisions

An orchestrator and specialised agents

Each agent is its own small Bun and Hono service with a few skills. It publishes an agent card at /.well-known/agent.json, accepts tasks over HTTP, and streams progress over SSE. Shared tools, like reading project files or running git, are exposed once over the Model Context Protocol (MCP) instead of being copied into every agent.

The orchestrator finds agents from their cards and health-checks them every 10 seconds; three failed checks take an agent out of routing. An LLM router picks a skill from the live list of online agents, and any answer that is not on that list is rejected instead of dispatched.

A supervisor that plans one step at a time

Requests with several steps go to a LangGraph supervisor. Instead of writing a full plan up front, it picks the next skill based on what earlier steps actually returned, and it can run independent steps in parallel. Two separate limits stop a runaway run. If a step that writes something fails without a clear reason, the run ends and asks the human to check the project, rather than assuming nothing changed.

Approval as a boundary, not a button

  • The approval gate sits outside the model’s control: the model never sees or produces the ID that approves an action.
  • Reject ends the run. Skip lets the plan continue without that step, and the same skill cannot be proposed again in that run.
  • Before a preview is shown, each write is classified as create, update, no-op or blocked. A no-op never writes anything.
  • DevOps files written by the model (Dockerfile, CI workflow, compose file, .gitignore) are validated before the preview: allow-listed base images and CI actions, COPY sources that exist, valid YAML, no undeclared secrets, no Docker socket or host-root mounts. A violation gets one retry with feedback, then the task fails.
  • Pending approvals are stored, so they survive an agent restart, and each one can only be used once.
  • The security agent’s model can only add commentary to findings the deterministic scan already produced. It cannot remove or downgrade a finding, and it cannot claim a CVE it has no source for. Both rules are enforced by the output schema, not just the prompt.

Visible while it works

The orchestrator emits AG-UI events: run and step start and finish, every tool call with its result and duration, and approval requests. The dashboard and terminal UI show what each agent is doing live, and every event is validated against the official @ag-ui/core schema before it is sent.

Spec-first, tested in CI

Every change starts as a numbered specification with a plan and, where needed, a verification record. CI checks the specification catalog, runs the type checker and the test suite, and runs a smoke test that real requests are dispatched end to end. A second workflow builds the standalone binaries for Linux, macOS and Windows, which are also published to npm.

Results

  • Finalist at DevOpsDays Cairo 2026, top 20 of 142 submissions.
  • Six agents behind one orchestrator, with human approval on every write and every command.
  • More than 140 specifications, about 1,600 tests and 318 commits in seven weeks.
  • One-command install with npx orchestrai or bunx orchestrai, or a single binary for Linux x64, macOS arm64 and Windows x64.
← All case studies