From agent sprawl to software factory

If you lead an engineering team, you’re already running a bring-your-own-agent shop, whether you planned one or not. One developer swears by Claude Code, the person next to her uses Codex, someone else lives in Cursor, and a new personal agent shows up almost every week. Asking everyone to stop and pick one would cost you the productivity gains you were promised. So each developer sets up their own skills, context files, and MCP servers, and connects whatever credentials they have on hand.

The cleanup falls to you. Every one of those setups is a black box: you can’t see which agents are running, what they can reach, or what they did yesterday. When one produces a great fix, nobody knows why it worked, and when one goes off the rails, nobody else knows how it was set up. The company gets the output of its agents, but no record of who did what and no way to learn from how the work got done.

A handful of companies have moved past this. Vercel’s factory authors 25 to 35% of its AI SDK team’s merged pull requests, Uber reports that more than 70% of its pull requests come from agents, and Stripe’s minions go from a Slack message to a pull request that passes CI with no human in between. They call these software factories, and they’re quickly becoming the goal for engineering teams everywhere. But few teams can see a path to that from where they are today.

Last week, we published the Secure Software Factory reference architecture to map that path, with our partners at Docker and Socket, as well as individuals from Barclays, Secure Theory, and other companies. It’s the first reference architecture built on Agent Baseline, and the group behind it continues to grow as more people contribute. Allie Howe, who led the working group for the paper, wrote up how it came together on X.

Why we started with the software factory

Agent Baseline was built to be put to work. When we published it with Docker and Snyk in July, Kamil wrote that it was meant to be applied to real systems, not used to certify ourselves. The software factory was the obvious first test: it’s the problem teams kept bringing to us, and one of the hardest to secure.

A software factory combines three things that are each risky on their own. It reads input outsiders can influence, like tickets and issues. It holds private data, like source code and credentials. And it acts on the outside world, opening pull requests and calling external services. Simon Willison calls that combination the lethal trifecta, and a software factory can’t drop any of the three without becoming useless. If Agent Baseline works here, it’ll work in most places agents run.

The software factory is the infrastructure, not the agent

Most teams start with the harness and the model. When they try to get out of the black-box state, the first question is usually which ones to standardize on. Those choices matter, but they’re only part of the picture: models and harnesses keep improving, and most organizations end up supporting more than one as developers use whatever tools fit their workflow.

What unlocks autonomy is the infrastructure surrounding the agents. The reference architecture describes it in two parts.

First, four boundaries that apply no matter which agent is running or what it decides to do:

• A sandbox that contains where the agent’s code runs
• A network boundary that controls which systems it can reach
• A supply chain boundary that admits only approved packages, tools, and sources
• Identity and access, which decides who the agent acts for and what it’s allowed to do on each task

Second, the outer loop. Every run leaves a record, and reviews, evals, and production results feed back into the agent’s shared setup, so what one person figures out improves the setup everyone else starts from.

The agent proposes, policy authorizes, and infrastructure enforces. The agent runs untrusted and without credentials in a sandbox, built from components the supply chain firewall checks. It proposes a scope to the runtime authorization service, which decides from trusted context (identity, task, delegation, and policy) within the bounds of the agent's profile, sends high-impact actions to approval, and stops anything outside policy. The egress proxy enforces the authorized scope, attaches credentials, and lets only authorized operations reach downstream systems. Observability records every grant, admission, process start, and request. The four boundaries, sandbox, network, supply chain, and identity and access, are enforced outside the agent's loop.

Together, that infrastructure is what lets you swap in a better model without rebuilding anything, share an agent across teams, debug it when it misbehaves, and give it more autonomy without giving up control.

What it looks like for one agent

Say a developer on the payments team builds an incident agent for the service her team owns. When an alert fires, it reads the service’s logs, finds the likely cause, and opens a pull request with a fix. Within a few weeks, the rest of the team depends on it.

But the agent still runs on her setup. It uses her skills and prompts, her MCP servers, and her credentials, which reach far more than the payments logs. If it opens five pull requests in an hour or touches a config file outside the service, nobody else can tell what it saw, what it was allowed to do, or why it acted. When she goes on vacation, nobody knows how to fix it. And when another team wants one for their own service, they copy her setup and tweak it until it works, so now there are two black boxes.

Making that agent part of a software factory doesn’t take a different model or harness. It takes the infrastructure:

• The agent’s skills, tools, context, and sandbox become a shared profile the team owns and versions, instead of one person’s configuration.
• Each run starts in a fresh sandbox, and its traffic goes through a proxy that only reaches the payments logs, the repository, and the model gateway.
• Its tools and packages come from an approved set, so a run can’t pull in something nobody vetted.
• The agent gets its own identity and never holds credentials. For each alert, it gets access to that service’s logs for the last hour and permission to open one pull request.
• Every run leaves a record of what triggered it, what access it had, and what it did.

Now when the agent goes off the rails, anyone on the team can see exactly what happened, and its access was limited to the task from the start. Evals across runs show where it struggles, and improvements to the profile reach every team using it. One developer’s experiment becomes a shared service.

Where to go from here

A shared service is the first big step. The reference architecture describes what comes after it in three levels of autonomy, defined by who directs the software factory’s work.

A developer starts every run at the first level, like in the incident agent example. At the second level, a predefined workflow runs the software factory as a shared service and starts work from tickets and alerts. At the third level, an orchestrator agent decides what work to take on and splits it across other agents. The four boundaries stay the same at every level; what changes is how much of the supervision people used to do moves into infrastructure. I wrote about each level in a LinkedIn article.

For the full picture, the Secure Software Factory reference architecture covers how each boundary works, how the controls deepen at each level, a walkthrough of agents handling a production incident, and how it all maps to Agent Baseline’s outcomes and controls. Version 1 is open for comment, so tell us where it doesn’t match how you build.

Keycard focuses on the identity and access boundary. If that’s the part you’re working through, book a demo.