Five days, five stops, 147 merges: our week with a model that had no name
You keep the human work. We build the rest.
Everything below comes from files we opened between 22 and 27 August 2026, Denver time: our dispatch ledger, our door log, and GitHub’s merge history across six repositories. The five-day week closes on Wednesday at 08:43, a partial day; the 24-hour trial reported at the end runs from Wednesday morning into Thursday.
On Sunday, 23 August, our fleet of coding agents dispatched 1,705 separate runs. Four out of five finished the work they were given. One pull request reached a production branch.
That gap is the most useful thing we found all week.
One rule held from the first launch. The only thing that counts is a pull request merged into a production branch, and an issue closed by that merge. Not code written. Not agents launched. Not a confident summary at the end of a run. Agents are very good at producing convincing output that never becomes a change anyone uses.
The week ran as a route. Five stops, each with a name, a breakage, and a lesson.
Stop one · Saturday
The instruments lied
One machine sat at a load of 98 with no idle time left. The obvious read is a runaway process. The real cause was our agent runner, opencode, snapshotting its whole working folder every turn; one agent’s pile had reached 21 GB. We turned snapshots off, cleared the piles, recovered about 205 GB of disk, and load fell from 98 to 9.8. The agents had been working the whole time.
We misread the instruments twice more that night. A batch of checks returned a network error, so we wrote the provider off as down; one agent tried again later on the same machine and got an instant answer. The error never meant gone. It meant too many at once, and our detector cut a healthy machine carrying 32 agents down to one. Then five throughput readings in twenty minutes gave 46%, 79%, 97%, 100% and 98%. Nothing about the fleet changed; every gap was our own measuring tool moving.
What we learned. A busy machine is not a stuck machine, and a settings change does nothing until the running processes restart. A rate is only as trustworthy as the definition printed beside it.
Stop two · Sunday
1,705 runs, one merge
At two in the morning, twelve background jobs were all set to wake every 300 seconds, and four wrote to the same work queue. The dispatcher that hands work to agents timed out seven times in four hours and refused to guess at a work list. Throughput fell from 51 items closed per hour to 3. The queue was never slow: run by hand seconds after a failure, it answered in 0.59 seconds. We moved every job onto prime-numbered intervals of 307, 331, 373 and 367 seconds, so they realign about once every 161,000 days.
By morning we found a deadlock of our own making. Every work item in our tracker, beads, carried the same owner label, and the step that claims an item failed permanently on anything already carrying it. Nothing could be claimed, so one item went to three agents at once and the stage meant to close finished work closed nothing. In the same batch, a safety notice we had added to every agent’s brief contained the exact words our outbound gate scans for, so the gate held all 40 real work items.
What we learned. Equal timers do not collide once; they collide forever. Never put a gate’s own vocabulary into the text that gate inspects. And never inherit a limit: the fleet had been held to 8 agents on an unmeasured number copied from an old report, and uncapped that Sunday it peaked at 102.
Stop three · Monday
The day our two numbers disagreed
Monday was the busiest day for the model itself, and our agent success rate barely moved: 80.7% on Sunday, 81.1% on Monday. Merges went from 1 to 25. The number we had watched all weekend could not tell a day that delivered nothing from a day that delivered 25 changes, because success only meant an agent had finished its own errand.
What we learned. When two numbers disagree, the one tied to delivery is the real one. Keep the success rate as a health signal and move the scoreboard to merges.
Stop four · Tuesday
Our worst day and our best day at once
Tuesday morning we found the bug that had been quietly capping us. The step that turns finished work into pull requests had its working directory fixed when the function was written rather than when it was called, so by default it ran in the wrong checkout, applying one project’s changes against another. Not one pull request had ever been opened against a second repository. Resolving the directory at call time fixed it, and the first such merge landed that morning.
Builds kept dying with a bare “shutdown signal.” We had been raising the memory ceiling for the type checker, which was exactly backwards. The build machine holds about 8 GB while a comment in the code claimed 16, so our ceiling outgrew the box and the operating system killed the runner. We measured five ceilings on the real machine instead. Eight gigabytes — measured, not guessed — let the build finish, and it has held since.
Tuesday was our worst day for rate limits, with 15.7% of calls refused, and our best day for delivered work, with 78 merges and 53 issues closed. When the free path pushed back, a second route through OpenRouter carried 2,457 of the day’s requests, roughly double its Monday share.
What we learned. Check the address, not the exit code: on Sunday an agent recorded success after writing output to a path that does not exist on its machine. Sometimes the fix for a starving machine is to ask it for less. A second route does not make a fleet faster; it makes its bad hour survivable.
Stop five · Wednesday
The codename comes off
The free window closed on Wednesday morning and the model stepped out from behind its codename. By eight o’clock the fleet had landed another 23 merges and closed 22 issues. The morning’s success rate dipped to 67.3%, and the dip is worth reading closely. Every failure was the same kind: an agent that ran, finished, and produced nothing under the name it was told to use. Provider errors had vanished.
What we learned. A narrower failure surface is a better one, because you can only fix what you can name.
What the route added up to
147 pull requests merged and 96 issues closed in five days, the last partial. The best rolling 24 hours held 89 merges.
Set that beside the agent count. Sunday ran 1,705 runs and landed 1 merge; Tuesday ran 659 and landed 78. The model never changed. Everything in between was pipe repair.
The numbers underneath
Across the window, 3,258 runs produced 2,612 successes and 586 failures.
That curve and the merge chart describe the same week and disagree about it. So we test the instruments too: we plant a fake failure now and then to watch the alarm go off, because a guard that has never fired looks exactly like a broken one.
The fleet made 135,401 requests in the logged week: 43,085 on Sunday with 91.4% served, 53,562 on Monday with 94.9% served, then 33,754 on Tuesday with 15.7% refused for rate limits. A typical answer took between 8.4 and 13.2 seconds.
Three windows hold 81% of the week’s throttling. The fleet carried on through all three, because a throttled route is now skipped rather than fatal.
How the system changed
What runs today is a conveyor, not a crowd. Work items are minted automatically from the state of each repository. A dispatcher reads how much room each machine actually has, picks work by score, and staggers the launches. Each agent gets one item, its own isolated folder, and no permission to push, deploy, or reach the open web. The five machines share one private network, Tailscale, so any can take a lane. A harvester collects the output, applies the patches to fresh checkouts, opens the pull requests, and puts them in a GitHub merge queue. The merge closes the issue, and the scoreboard moves on its own.
Around that chain sit the parts we built after being burned. A health probe every fifteen minutes repairs what is mechanical and files a ticket for anything needing judgment. A monitor watches the monitors, and exactly one dispatcher runs. Our biggest source of new work turned out to be recycling: an item closed while its issue stayed open, with no pull request to show for it, gets reopened with a note to try a different angle.
What Ox Alpha turned out to be
On Wednesday, Z.ai introduced GLM-5.3 Flash and said it had been “previously previewed as Ox Alpha.” An engineer there added that Ox Alpha was an early version and the release build is more stable. Its name appears in our request log for the first time that morning, and it is a paid product now.
Coverage keeps mixing two models: GLM-5.3, the flagship Z.ai released on 14 August, is not GLM-5.3 Flash, which it released on 26 August. Ox Alpha was Flash.
Z.ai describes Flash as a natively multimodal mixture-of-experts model: 320 billion total parameters, 18 billion active per token, a 1,048,576-token context window. It is MIT licensed, with weights public on Hugging Face at roughly 328 GB. Standard pricing is $0.15 per million input tokens and $0.50 per million output; OpenRouter lists it at half that and flags the discount. Z.ai’s benchmark table puts Flash at 84.3 on Terminal-Bench 2.1, against 85.0 for Claude Opus 4.8, 85.8 for Gemini 3.7 Flash and 87.4 for GPT-5.6 Terra, and well ahead of its own predecessor on agent work: 63.4 against 46.2 on DeepSWE, 48.8 against 26.2 on AutomationBench. Z.ai also says the preview week ran entirely on domestically produced Chinese chips.
Those are vendor numbers and a vendor claim. We ran none of those benchmarks and found no independent verification of the chip claim. Our own reading is narrower: on a repaired pipe, this model turned 659 runs into 78 merged pull requests in a day. That is the only benchmark we are qualified to report.
What we now assume
The free window closed on Wednesday morning, so we ran the comparison. We took a real work item from the Ox Alpha queue and handed it, unchanged, to our in-house tier: a maximum-reasoning configuration on seats we already pay for, which we call Luna. Same brief, same rules, same bar.
Luna came back with the same artifact discipline: a real diff, acceptance checks that run without pulling in extra dependencies, an adversarial case among them, and assumptions written down instead of hidden.
The interesting part is where the two disagreed. The Ox Alpha lane had judged the work already done and proved it. Luna shipped a small hardening anyway. Both answers are defensible, and that is the finding. Two competent engines can reach opposite, correct conclusions about whether a thing is finished, so the verify stage must stay independent of whichever engine produced the work.
Here is our assumption, argued from the week’s own numbers. The same model produced one merge on Sunday and 78 on Tuesday. The engine held constant; everything that moved was ours. That makes engine choice a smaller lever than pipe quality, worth testing but not worth agonizing over.
Comparable artifact quality at one work item is all we can claim today. A sample of one is not a verdict. A 24-hour Luna trial is on the same queue right now, judged by the same scoreboard: merged pull requests and the issues they close. We will publish what it says.
What happened when the free model disappeared
We said we would publish what the 24-hour trial said, so here it is. The free window closed on Wednesday morning. We put our own tier on the same queue — the maximum-reasoning configuration on seats we already pay for, the one we call Luna — with the same rules and the same scoreboard: pull requests merged and the issues they close. Six repositories, Denver time, from 9:50 on Wednesday morning to 9:50 on Thursday morning.
50 pull requests merged. 47 issues closed. Around 45,600 net lines of code. Open issues across the six repositories fell from 97 to 56, and one repository went from 32 open to 7.
Now the number that matters more. 467 lanes resolved. 211 reported success. 184 produced a real code patch. 50 of those merged. Four lanes said “done” for every one that reached a main branch.
That gap is not all waste, and it is worth saying exactly why. A lane can finish clean and still make a patch nobody should land: the issue was fixed upstream while the lane worked, the diff no longer applies, the checks come back red. Only one duplicate actually landed, forty-three minutes after another patch had already closed the same issue, and it landed because the loop drained banked work without asking whether the issue was still open. That check now runs before a lane starts. It does not yet run before a patch lands, which is the same check in a second place.
Set that beside the week above. Ox Alpha’s best full day was 78 merges and 53 issues closed, on a pipe that had been warmed up all week. Luna’s 24 hours were 50 and 47, run through two load storms and a machine expansion in mid-flight, on seats we already pay for. Different conditions, comparable output. That is the honest shape of the comparison, and it is as far as one day of each will carry it.
What we learned. The output number is 50 merges. The efficiency number is 24%. Only one of the two tells you what to fix next.
Capacity is a property of machines, not accounts
The trial cost us two outages in a day, and both were the machine rather than the model.
The first came in the early afternoon: a burst of repository clones filled the system’s open-file table, and the failure killed the pull-request drive part-way through applying a patch. The second came late that evening, when 49 lanes stacked their test runs at once and the one-minute load average reached 221 on 32 cores. Through both storms the model answered normally and the queue stayed full. The machine simply could not run what it had been handed.
So we measured the ceiling instead of guessing at it. It sits at roughly fifteen to twenty concurrent lanes per machine, and what sets it is not model calls — it is test runs. Past that line lanes slow each other down three to four times over. You buy nothing and pay for it in wall-clock time.
Which is why we stopped adding accounts. We run five against one shared quota now, and that spreads the bill without lifting the ceiling by a single lane. Accounts buy tokens; machines buy throughput. So the fleet grew sideways instead. Above a measured load line, new lanes go only to a second machine while the local ones drain, and each remote lane opens one connection and does its whole job through it. A parallel burst of eight clones had been quietly dropped by the far end’s connection limiter, and we only caught it because the errors it produced looked like missing repositories.
The bill: about eighteen percentage points of one weekly quota went in the first ten hours, before the work spread across accounts. We also capped retries at three. Past three attempts a lane is not learning, it is spending.
Adding a fifth account did not make the fleet faster. Adding a second machine did. The engine is a smaller lever than the box it runs on — and the box will tell you its real limit, if you agree to measure it instead of inheriting it.