Software factory or agent team? Two questions decide it


TL;DR: The software factory vs. agent team debate asks whether an agent is a part you program or a teammate you manage. I went back through a year of my own published trials and that question didn’t predict what worked. Two others did: can you write the acceptance check before the work starts, and how expensive is it to be wrong? Those give four cells, and I’ve shipped something in each one. The factory wins the top row, with one catch: a check that exists is not a check that sees. The role-based team wins no cell in my data, and I say that having never run one.


My feed spent this week on one argument. One camp says build a software factory: work comes in through a queue, agents are stations, gates decide what merges. The other says build an agent team: agents hold roles and you coordinate them.

Under the labels, it’s a question about what an agent is. Is it an engineering primitive you program, or a teammate you manage?

The sensible answer is that each shape has its ideal use cases. Fine, but which ones? I have a year of published runs, so I went back and checked.

The two questions

When I lined up what worked against what didn’t, the agent’s nature never came up. These did:

  1. Can you write the acceptance check before the work starts?
  2. How expensive is it to be wrong?
A miss is cheapA miss is expensive
You can write the check firstA loopA pipeline with adversarial verifiers
You can’t yetExplore, with as many agents as you likeDesign it yourself, then move up a row

Here’s one real run per cell.

Check first, cheap miss: a loop

Earlier this year I pointed a script at a failing spec in Forge and went for coffee. When I came back the spec was green and a commit was waiting for review. One agent, a five-iteration cap, and bundle exec rspec as the stop condition.

That’s a factory station at its smallest. No roles, no ceremony. Months later, a Codex agent that had just finished a full pipeline run gave me the rule for this cell: for routine work, the ceremony and the token cost outweigh the benefit.

Check first, expensive miss: a pipeline

Key rotation for reeve’s audit ledger had a frozen spec and a brutal cost of error: epoch records are immutable, so a bug there is permanent. I ran it through a pipeline by function: one builder, then a fan-out of verifiers who each got claims to break.

They confirmed nine findings before merge. By the builder’s own admission, three would have shipped from a normal single-session build. (There was no control run, so that’s a self-report.) The task ran about 1.7× its budget, and the builder guessed roughly 3× the cost of building it directly.

This is still a factory. But look at which stations earned their cost: the ones arguing with the output, not the ones producing it.

No check yet, cheap miss: explore

This is where many agents pay. A cold-reader agent got only framesmith’s 1,278-line README and twelve tasks, and in about eighty seconds it found that the README’s own examples fail the parser they document. Scouts pay in the same place: upstream docs, the ecosystem, repos I won’t touch.

One honest note. Scouts inside my own repo were waste: 3.3× their budget, and I still had to read every file. Exploration pays when the context is foreign to you, not when you’ll have to read it anyway.

No check yet, expensive miss: design it yourself

Before any agent touched reeve, I designed the kernel by hand: six small classes and four collaborator protocols, frozen under a git tag. That took longer than either agent-built module took to build.

Neither a factory nor a team helps in this cell, because parallel agents don’t discover an architecture. They consume one. The good news is that this cell is temporary. Once the interface is frozen, you’ve written the check, and the work moves up a row.

So where’s the agent team?

Nowhere in that table, and I owe you the reason. I have never run a role-based team (PM agent, developer agent, QA agent). I read the evidence against it and built the other shape. So no cell in my data goes to the team, and that tells you about my data as much as about teams.

The closest thing I have is sessions that file bugs on each other. Each session owns one repo. What crosses between them is an issue, never context. One of those loops went from the second issue filed to the release merged in four hours and thirty-eight minutes.

I don’t program those sessions like primitives, and I don’t manage them like people. They’re owners that exchange artifacts. If the debate needs a third option, that’s mine.

The catch: a check that exists is not a check that sees

Deterministic beats clever, and I’d make as much of a gate deterministic as I can. But the most deterministic gate I’ve studied blocked nothing in eight runs. It watched the Edit and Write tools. The agents made 398 Bash calls against 8 Write and 3 Edit, and wrote most files through Bash. Nobody cheated. The gate was deterministic and blind.

The same goes for the factory’s favourite rule, “tests green, merge.” The 37-agent review of reeve found six silent fail-open paths. None of them crashed. None failed a test I would have thought to write. And when I reverted each fix to confirm its regression spec went red, three of ten specs stayed green. They were covering nothing.

So where to spend your tokens has a number in my data. That review cost about 1.5M of the build’s 1.9M agent tokens, and it was worth more than everything else combined.

What I’d tell past-me

  1. Pick the shape per task, not per company. Reeve’s kernel and reeve’s key rotation live in the same repo and needed different shapes.
  2. If you can’t write the check, you’re not ready for either shape. Design first, by hand, and freeze it.
  3. Spend on the stations that argue. Fan out for verification. Keep writes single-threaded.
  4. Put the guarantee where the agent can’t reach it. Merge time, a required review, a spec you’ve seen go red.
  5. Skip the ceremony when a miss is cheap. A capped loop and a test suite is a complete system.

Caveats

  • One person, my own repos. My agents are on both ends of every handoff.
  • Small samples and no control runs. The pipeline numbers are four trials. “Would have shipped” is the builder’s word.
  • No role-team trial. My case against role-based teams is other people’s research plus my own results with the opposite shape.

The debate asks what an agent is. I’d rather ask what I can check, and what it costs me when the check is wrong.

Victor Velázquez is the COO of MagmaLabs and the author of The Agentic Engineer — twelve durable principles for building production systems with LLMs in the loop.