A doorway shaped like a phone screen opens onto a long dark tunnel. A small figure waits on the threshold beside a lantern that has gone out, while two robots walk into the tunnel with a tape measure and a stopwatch, marking where it should end. Editorial illustration in a risograph print style: confident ink linework, two muted inks (deep slate blue and warm ochre) on an off-white paper ground, visible grain, generous negative space. No text, no letters, no numbers, no logos.
Last week we compared a debate between two models with one strong model answering alone, on a memory problem in a Python service. We called it an anecdote with numbers attached, and it was. One case can't tell you whether the pattern holds. So when a very different problem came up a few days later, we ran the comparison again.
The pattern held: the debate caught what the solo plan missed. But this case also shows the cost more plainly than the first one did. The debate made the fix better, and it made it much bigger.
The problem
Our phone app has a remote shell screen. It opens an SSH session, over the app's own connection to a private network, into a tmux session on a home server. Sometimes that screen sat on “Connecting…” forever. Worse, it sometimes said “Connecting…” when no connection attempt had started at all. An attempt that never opened had no overall time limit and no stated reason. Each step had its own timeout, but nothing ended the attempt as a whole, so the user waited and the log didn't say why.
The experiment
The setup differed from the first case in one way that matters. Last time the solo answered the debate's own questions. This time it wrote its whole implementation plan first, before any questions existed, the way an agent normally works when nobody asks it anything.
- Solo. The agent that would implement the fix read the code and wrote its own plan, in a place the planning and debating agents couldn't read.
- Debate. A planning agent wrote the task's plan with five open questions, and a separate read of that plan added a sixth. Then came a debate walk: Claude Sonnet 5.5 presented each question and MiniMax M3.1 Flash Preview argued it. The debate decided all six and raised three more of its own, so nine in all.
We scored the solo plan against those nine questions: did it address each one, and was its answer the one that held up?
Nine questions, one row each
| Question | Solo plan | Raised by | Live bug found |
|---|---|---|---|
| What the screen says before an attempt starts | Covered, wording overturned | Planning | — |
| Log the “not started” state? | Covered, decision reversed | Planning | — |
| What ends an attempt that never opens | Covered, same shape, three defects | Planning | — |
| An end that arrives with no reason | Missed | Planning | — |
| A session that opens, then goes silent | Missed | Planning | — |
| An attempt the user walks away from | Missed | Plan read | Yes |
| Events lost while the app's UI is gone | Missed | Debate | Yes |
| One attempt id on every log line | Missed | Debate | — |
| The wait for the terminal's first size | Missed | Debate | — |
The solo plan addressed three of the nine. It missed the other six entirely, and two of those turned out to hide bugs that are live today.
Where the solo plan held
On the central question it was right. It proposed two ends for a stuck attempt: a short limit on the native side, extended when the server shows an approval prompt, and a longer backstop in the app for anything the native side can't see. The debate chose the same shape. The solo plan also doubted that its timing values should be hard-coded, and the debate agreed. They became settings.
Where it didn't
1. The right shape, with three defects in it
The debate accepted the two-ends design and then attached three conditions to it, each traced to specific lines of code. Each one is a bug the solo version would have shipped.
- The backstop's reason would have been erased. The app sets its own reason (“the machine did not answer”) and then closes the session. But closing makes the native side send a “closed” event with an empty reason, and the screen overwrites its text with whatever the event carries. The user sees no reason, and the log records an empty one: exactly what the fix was meant to prevent.
- Its time budget was a hand-added sum. The backstop's limit was written as one number, the native limit plus the worst case before a session exists. That worst case is already the sum of three named constants. A second, hand-typed copy goes stale the first time one of them changes. The decision was one function that adds up the named bounds, with a test that pins the total.
- A late answer would have leaked a session. If the backstop gives up before the native side has returned a session id, and the id arrives afterwards, it has to be closed at once. Otherwise a tmux session stays open on the server while the app believes the attempt is dead.
The planning agent's facts also corrected the solo plan's arithmetic. Its worst case left out a 15-second wait for a lock and a 5-second status read, about 20 seconds in all.
2. Two of its three answers were overturned
For the screen before any attempt starts, the solo plan proposed “Preparing the terminal…”. It was ruled untrue. Nothing prepares the terminal in that window. The app is waiting on a size measurement from the layout, which may never arrive. The honest line names what it's waiting for.
On logging the “not started” state, the solo plan said not to, because the line would repeat. The debate counted the call sites: about once each time the screen opens. So the line costs almost nothing, and it's the only record of an attempt that never began.
3. Two live bugs off its path
Deciding what happens to an attempt the user abandons, the debate found that leaving the screen mid-connect already orphans a tmux session on the server. The session id arrives after the screen is gone, and nothing closes it.
Deciding what happens to events sent while the app's UI isn't running, it found that the native module drops them. One of those events carries the device's network identity, which the app has no other way to read. If it's dropped, the identity can be lost and the phone has to be approved on the network again.
Neither bug is about a hung connection. Both were found by asking what else crosses the same boundary.
4. The rest
The other misses were less dramatic, and they're the kind that make the next incident shorter. Every way an attempt can end now has a named kind with its own reason, instead of an error string copied across three languages. That also exposes “tmux is not installed”, which used to look like a clean disconnect. A session that goes silent is now noticed by a keepalive probe. Every log line for an attempt carries the same id, so a copied log can be read one attempt at a time. And the wait for the terminal's first size has a limit, after which the screen says so and offers a retry.
Where the catches came from
The first post couldn't separate the debate from the planning that came before it. This case can, a little. Before any debate ran, the planning agent's questions and facts already covered three of the problems: the premise behind the logging decision, the empty-reason problem and the silent session. It also corrected the time budget. A read of the plan added the abandoned attempt. The debate raised the last three questions itself and attached every one of the binding conditions above.
So a good part of the gain comes from writing questions down before answering them, not from the second model. The debate adds the other part: the conditions that turn a correct shape into correct code.
What it cost
This is where the case differs most from the first. There, the debate made the fix smaller in one place: one read path instead of two. Here it made the change much larger. The solo plan was a change to the app's screen and the native timeout. The walked plan touched Go, Kotlin and TypeScript, added hidden settings, and landed as about 1,300 added lines across 17 files.
The process wasn't free either, and it wasn't self-checking:
- The debate took about half an hour. It ran without a judge, and the check that compares its decisions with each other didn't finish.
- The plan read afterwards found five gaps between the decisions: which kind each new end carries, where the settings live, how often an automatic reattach may happen. They were decided directly, with no second debate. So the plan grew from nine decisions to fourteen.
- Review turned the first implementation back with two findings. One was a bug the debate itself introduced: a terminal size that arrived after the new size limit had expired never started the attempt. The other was a gap neither plan named. The steps after login (opening the session, the terminal, the tmux start) had no limit. Both were fixed in one round, and the second review accepted it. From launch to landing took about two hours.
The build is now on the phone. The shell connects and reconnects. The failure paths are covered by tests. They haven't yet been met on a real device.
The two cases side by side
| Memory problem | Hung connection | |
|---|---|---|
| Solo arm | Answered the debate's questions | Wrote its own plan first |
| Questions the debate decided | 7 (5 + 2 it raised) | 9 (6 + 3 it raised) |
| Solo agreed on the core | Yes: churn, not a leak | Yes: two ends for a stuck attempt |
| Solo missed entirely | 2 | 6 |
| Live bugs found on the way | — | 2 |
| Gaps a plan read caught afterwards | 1 contradiction | 5 gaps |
| Effect on the size of the change | Smaller in places | Much larger |
The honest limits
- Two cases are still two anecdotes. They point the same way, and they're still not a study.
- Not every catch is worth its cost. Some of what the debate added, the attempt ids and the hidden settings, is discipline a team might reasonably choose to skip on a phone app's shell screen. We didn't measure which additions mattered. We only know which ones found bugs.
- The solo planner was also the implementer. That is the realistic setup, but it means the solo plan was written by an agent that expected to do the work. It may have scoped the work to fit.
- We set one sibling case aside. We ran the same experiment on another task, but there the solo plan was written after its author had read the planning agent's questions. Its answers were primed, so it isn't counted here.
- Effort wasn't matched. The debate arm spent far more time and than the solo plan. Controlled studies find that much of what debate gains comes from that extra effort, and that taking a majority over several answers does nearly as well (see the first case's limits). A solo arm given the same budget is the comparison these two cases don't make.
So, is it still better?
Yes, but the answer is less one-sided than last time. The solo plan was a competent answer to the question it asked itself, and it got the central design right. Everything it missed was a step off that path: an event arriving after the screen is gone, a reason overwritten by the next event, a constant copied by hand. Those are the same kinds of misses as in the first case, found the same way, by looking from somewhere else.
What changed is the price. When a debate keeps finding things, the change grows, and a bigger change brings its own bugs. One of ours came from the debate's own decision. The debate needs a step after it that reads the whole plan, and a review after that. It also needs someone willing to say that a catch isn't worth what it costs. Two models found what one missed, and it still took a person to decide how much of it to build.