What makes a good software factory

Give a hundred developers Claude Code and you get a hundred people producing a lot more code. Does your product get to users any faster, or in better shape? Mostly not. The code piles up in review, and the old handoffs stay exactly where they were.

How much an agent can do on its own depends on the team around it. With good tests and good context, a developer can let it run for an hour. Without them, the same developer sits next to the agent and corrects it every few minutes.

Either way, a semi-autonomous delivery system looks different. There, agents finish whole changes on their own. And when I correct my agent, yours gets the correction too.

So here we are: everyone is building a factory now. Good!

But I keep watching the same thing happen. Engineers lift their work one level up (as we told them), into the factory, and bring the old sins along. The spec still gets written upfront like it’s 2005. The people who understand the domain are, once again, not in the room when the factory gets designed.

That’s a shame, really, because the factory is probably the biggest lever we ever had to scale proper product thinking into a delivery system that runs partly on its own.

Here are the twelve things, let’s call them principles, I look for.

1. Run one shared line
2. Design autonomy into the system
3. Keep one trail for every change
4. Build variants, let people choose
5. Escalate intent to the ticket
6. Approval by risk class
7. Never review your own work
8. A person owns every change
9. Keep the system understood
10. Stop the line
11. Measure how fast the factory learns
12. Fit the organisation, plan the removal

Why a hundred laptops aren’t a factory

1. Run one shared line

“But we already share our AGENTS.md!” I hear that a lot. AGENTS.md is the instruction file every agent in the repo reads, and often there’s a review bot in the pipeline as well.

And that’s a good start. It gives everyone the same rules.

But it doesn’t tell you why a rule is there, or whether it ever fixed anything. And when something goes wrong, there’s no single place where you can stop the work and fix it for everyone. You still have a hundred separate setups, just with the same file in each.

Here’s what that looks like in practice: An agent keeps getting a domain rule wrong, and I correct it, mostly by prompting, in my own session. Maybe my correction makes it into the shared file, maybe it doesn’t.

Even if it does, nobody can check later whether it helped. I know what happened, because I fixed it. You don’t.

I wrote about this in May, and it hasn’t gotten better since.

2. Design autonomy into the system

If you read every change your agent proposes, line by line, you’re working in a tight loop, and it doesn’t matter how good the tool is. Perfectly fine for highly risky or exploratory work!

It just doesn’t scale. Put a hundred people in tight loops and what you get is a very, very long review queue.

The alternative is a lights-on factory. The lights stay on because people still work in it. Agents implement larger changes on their own, and independent checks (both deterministic and agentic) give the agent feedback on different levels. People still decide whether the agent got the right assignment, and whether the result is good to ship. And the team agrees beforehand where those decisions happen.

That autonomy comes from the setup around the agent. You won’t get it from a better model alone, at least right now (we get to removing parts of that setup again in principle 12).

So let’s say review is no longer your bottleneck. Then integration is. Every change has to be built and tested before it can go out, and that takes time and machines. If your build takes 45 minutes and you have three machines to run it on, you get about four changes through per hour, and the best reviews in the world won’t make it five.

But most organisations get stuck much earlier. They just don’t know clearly enough what they want built.

That is intent: what someone actually wants from a change.

Agents can help a lot here. Give them a messy workshop conversation and they turn it into a first spec, and they ask the questions nobody asked in the room (there are always a few). They build variants that people can click through before anyone commits to one. And once a change is live, the team can use what happens in production to decide what to build next.

The factory has to support all of that. And engineers can’t design that part alone, because they usually aren’t the ones running the workshops or deciding what comes next.

If only engineers design the factory, it starts at the ready ticket. Figuring out what to build stays somebody else’s problem, like it always was.

3. Keep one trail for every change

Your AI tool has a usage dashboard, and it shows you who burned how many tokens. Nice to know, I guess, but it’s just activity.

Ask it what happened to a single change on its way to production and it has nothing.

A factory records exactly that, for every change, from what the product owner asked for to everything that happened until it went live. Every decision comes with its reason. So months later you can still look up who decided what and why, and what the change actually cost, long after everyone involved has forgotten.

And that history turns out to be useful for something else, too. It helps you find the factory’s own mistakes.

If the same problem shows up in twenty changes, you can trace it back to its cause and fix it there, once. Without the history, someone keeps fixing it by hand, one change after the other, twenty times.

Why most factories stay engineering tools

Most factories I see get the shared rules and the record of each change right, eventually. And then they stop. The people responsible for the product still have very little say in how the factory works.

John Cutler recently wrote about the software factory as a kind of identity defense, with engineers moving up a level to keep authority over the means of production:

Most of these emerging ‘factories’ seem to stop at the boundaries of existing professions.

That matches what I’ve been saying since the book launch in June and again in September.

Whoever builds the factory decides what it checks. So when engineering builds it alone, the test suite decides what’s good enough, because tests are the check engineers know.

By the time product and domain people get involved, those decisions have been made. They get the factory handed to them like any other tool, here you go. And their judgement about whether a result makes sense for the product never becomes one of the checks.

So product, domain and engineering people have to build the factory together, from the start.

4. Build variants, let people choose

Say “factory” and people think of car plants, and the best-known one is Toyota’s. Its production system is where lean manufacturing comes from, and a lot of what we believe about good processes today goes back to it.

On Toyota’s line every step is repeatable, so every car gets built the same way. New ideas do get tried out, but earlier, in development.

A software factory needs the same discipline in how it runs. Every agent run that produces a change uses the same shared, versioned setup, and nobody gets to tweak their own copy, however tempting that is.

But the product is different. In software, every feature is a new product decision, and agents make it cheap to build several variants as running software, in the actual design system and close to production data.

The people who know the product and the domain click through them. They might pick one, or combine the best parts of several into a better solution. That’s a much better basis for a decision than a paragraph in a ticket. It also leaves the big upfront spec with very little to do, which I can live with.

Keep the variants you dropped, and write down why. Some were just wrong. But others lost a trade-off that may well go the other way next year, and then you can pick them up again.

I argued in September that building is part of specifying, and this is how the factory lets your teams actually practice it.

5. Escalate intent to the ticket

People decide what they want from a change, and that means the goal and the limits the change has to stay within. Agents can do that too, but let’s focus on human-owned factories for now.

So what should an agent do when it isn’t sure? It asks a person, and it asks in the ticket.

Before it asks anyone, though, it looks up what it can find on its own, in the code and in earlier tickets. People only get the questions nobody else can answer, like what the goal is or how much risk is acceptable. They have enough of the other kind already.

Why the ticket? Because that’s where the product owner and the rest of the team work, and because a chat with the agent is gone once the session ends. And while the question is open, the agent keeps working on everything that doesn’t depend on the answer.

Sometimes nobody can answer right away, and that’s fine. The ticket then says the point is still open, and it names the person who can decide it.

What we want to avoid is the agent filling the gap with its own guess. For an agent, a task is a search for a solution inside the limits we describe. There’ll always be gaps in that search space, and that’s OK. But agents guessing all over the place? Most of the time nobody needs that, and nobody wants it. The trouble is that a single wrong guess doesn’t cause an error anyone would notice, so later runs quietly build on it, until someone finds out that the software does the wrong thing.

6. Approval by risk class

How carefully should you check a change? Well, look at how much damage it can do.

A typo fix in the docs needs a quick look, and a change to the auto-updater needs every check you have, plus a few you wish you had. It doesn’t matter whether a senior engineer or an agent wrote either of them.

You may know the author and trust them, but that tells you nothing about how risky this particular change is. Anything you can’t undo goes into the highest risk class, no discussion.

So it pays to make changes easy to undo. If you can switch a feature off without a new release, or roll it out to a few users first, a mistake does little damage. And then some changes can ship without anyone reviewing them at all!

7. Never review your own work

Would you let an agent grade its own homework? It’s pretty tempting. The agent that wrote the code is right there, and it already knows everything about the change.

That’s exactly the problem. It also brings along every blind spot that went into the change.

So the check that lets a change through runs with a fresh context, at the very least. That’s a new agent session that only gets to see the change, and none of the reasoning that produced it. For riskier changes we’d go further and use multiple layers of adversarial reviewers, each taking a different role, sometimes from a different model provider, or a person.

There’s a second trick I like a lot: The grader keeps some test cases back, and the agent writing the code never gets to see them. If it did, it would optimise for exactly those cases and miss the actual goal. Agents are very good at passing the tests you show them. So are we!

The tests and linters an agent runs while it works catch a lot. But they don’t catch slop. Slop is code that works and passes every test, and still misses what the team considers good. Maybe it’s far more complex than it needs to be, or it just ignores the patterns the rest of the codebase follows.

The same thing happens on the product side. A feature does exactly what the ticket says and still feels generic, or it doesn’t fit the domain.

Engineers can write checks for the first kind of slop. The checks for the second kind have to come from product and domain people. And factories built by engineers alone tend to leave those out.

8. A person owns every change

Even when a change ships automatically, somebody is responsible for it. Either you approved the change yourself, or you’re responsible for the rule that lets changes of that kind ship without you.

If nobody’s name is on it, nobody looks. And then the factory ends up approving its own work. You do want this in a dark factory, where no person approves changes anymore, but in a lights-on factory you’ve got plenty of other working modes.

Over time you can approve changes with fewer checks, as long as reverts and incidents stay low. And if they go up, the factory can switch back to stricter checks right away. Nobody should have to wait for the next meeting to bring it up.

If you save an hour of review and then spend two hours fixing errors, you haven’t saved anything.

9. Keep the system understood

Before a change ships, someone reads the test results and the review comments. So now they know whether the change works.

But does the team still understand how its own system works afterwards? Test results and review comments don’t tell you that. It’s a different check.

Geoffrey Litt gets agents to build small interactive models so he can explore how the code works. He also has them quiz him on the changes. It’s easy enough to nod along while the agent explains.

And Maggie Appleton asks why we keep making people read walls of agent-written text. Give the team something they can explore together, and let the agent do the work of building that view. That belongs in the factory too.

You can let go of quite a lot here, on purpose. The team decides what it still needs to understand, and the factory gives people the explanations and tools they need.

10. Stop the line

A factory has two jobs. It builds the product, and it gives people and agents what they need to improve the factory itself: evaluations that show what went wrong while building, and the means to fix it. What actually gets changed is still their decision. We don’t have recursive self-improvement yet, whatever you’ve read this week. And both jobs need someone responsible and a backlog.

When the factory produces a defect, stop the line! That’s the habit I’d copy from Toyota first. Any worker on Toyota’s line can pull a cord when a part comes out wrong, and production just stops.

In a software factory, that means you find the place in the factory where the defect came from, and you fix it there. Then, before more work goes through, you show that the fix catches the original mistake. Otherwise the same defect turns up again in every change that follows, right on schedule.

But some problems only show up over several weeks. For those, the factory reviews its own results at regular intervals and suggests fixes. A person decides which to pick.

11. Measure how fast the factory learns

How long does it take you from spotting a failure to having a proven fix in the factory? That’s the KPI I’d set first.

A fix only counts once similar work goes through without the same failure. Until then it’s just a hope. And if it takes long to get there, the failures simply aren’t reaching anyone who can do something about them.

I’d also watch what it costs to produce a result that has passed its checks. Just don’t make it a target!

Because as the factory gets better, the team takes on harder problems, and harder problems cost more. A cost target would punish exactly that.

And counting pull requests per week tells you nothing, although it does make a nice chart. Most of the time, you should stop doing it.

12. Fit the organisation, plan the removal

The factory runs on systems the team already owns, or picks itself, and the architecture of the product stays the team’s decision.

If only an outside partner can change the factory, the team depends on that partner, and once the partner leaves, nobody can maintain it. And partners do leave, that’s the whole idea. Now, it’s perfectly fine if a customer doesn’t want to own software development as a business capability. But then make that discussion explicit. For me the handover is done when the team has improved its factory once, without help.

There’s one more thing I’d build in from day one. For every rule in the factory, write down whether it only exists because today’s models make mistakes.

Those rules are workarounds. So you take them out once the evidence is there that newer models don’t make these kinds of mistakes anymore. Permission boundaries and the evaluations themselves stay, however clever the next model turns out to be.

Removing rules is part of maintaining a factory, just like adding them!

So what about the laptops?

Let’s close this by getting back to the 100 developer laptops. None of this makes the laptop obsolete. Highly risky or exploratory work stays in a tight loop, and trying things out locally is fine, go ahead. But the changes that are meant to ship, and the checks that decide on that, run through the shared factory.

The factory costs something, of course. Someone has to own it, and the team has to take time away from features, on purpose, to improve it. Try putting that on a roadmap.

In exchange, the people who understand the product get to help decide how it gets built. And when I stop the line for a defect, the fix applies to every later run, for everyone, and that includes yours.

With a hundred separate setups, each of you has to learn that lesson on your own. And you’ll keep scrambling with shared skills, weekly best practices meetings, prompt tips in a Teams channel and a wiki page.

Thanks to Roman Stranghöner at neureif for the feedback.