Townsfolk · Builds 9 and 9.5

The brain remodelled

One overloaded thought split into a mind and hands, then tested in several ways, including six makers' models on the same trial.

▶ Watch the Sky Trials: six models, one riddle (video)

← Townsfolk: the idea and every build

Why

In Build 8, one thought carried everything she might need: about 40,000 characters, 57% of it descriptions of her abilities, and one action in four failed. The model was being asked to understand the moment, decide what to say, and pick the exact ability and arguments for every step, all at once.

What changed

Build 9 split that thought into roles, and moved the telling of the world to code. The scene is written by the server from the game's facts: the blocks around her sorted into standing things, laid ground, trees and water, with handles such as chest-1. System 1 waits in code and makes one small call to decide who answers. Her mind (System 2) has a small context: it says her line and states what she means to do in plain words. Her hands see every ability and its arguments, and turn what she means into steps for the body. System 3 does slow work beside all this, never on the way to an answer.

The new chain replaced the old path outright, with no switch back.

The gamefacts about the worldScenewritten by codeSystem 1who answersMindSystem 2: her line,what she meansHandsevery ability:steps for the bodyBodyacts in the gameSystem 3slow work, never on the way to an answer
The gamefacts about the worldScenewritten by codeSystem 1who answersMindSystem 2: her line, what she meansHandsevery ability: steps for the bodyBodyacts in the gameSystem 3, beside the mindslow work, off the answering path
Build 9's brain. Grey parts are code, blue are small models, gold run on the companion's own model.

How it's tested

No single test catches everything, so Build 9 is tested in several ways, most of them built in Build 9.5. All run on copies, never on my own world or on what Robin and Storm really keep. The evals below use them.

  1. The play-test rig. A copy of the world and of her folder, played live. It can take pictures of the screen, press "Show what I see", and quit the game the way Quit Game does.
  2. The same half hour. Twenty-six minutes of me, scripted and timed: a greeting, a question with a true answer, a small job, a long job with a question in the middle, something that can't be done, three minutes of silence, a stop, being shut in, nightfall, and a day left to her. Every brain is played against it.
  3. Replays. Real thoughts from a play record put again to real models, at other models and settings. The last replay passed 24 of its 25 situations.
  4. System 1's judgement. 29 moments marked by hand with who should answer, and in what order. It was right in 22 to 24 of them across the day's runs, with every stop right and nobody missed.
  5. Checks. 258 automated server checks, and all 28 starting scripts and rules pass the game's own validator.
  6. Critics. Six agents read the code and the play record and wrote ranked findings.
  7. A bench for models. Recorded thoughts put to each candidate model as her mind and as her hands, measuring whether the answer is one the server would accept, the seconds and the cost.
  8. Same task, same words. The Sky Trials: one task given to six makers' models alone, then each interviewed, then all six together.

The evals, in order

  1. Part 1 · Build 9 · 15 November, evening

    The first evening, and a night of fixes

    The evening's play was recorded, and six critics then read the code and the evening's 191 MB play record. Their headline: 40% of the things a sister tried to do didn't do what she meant, and two thirds of those failures were told to her as successes.

    A live play-test rig was built so a fix could be proved in the real game, on a copy, without me. It found what no critic could, starting with a crash in its first ten seconds. About a hundred fixes went in overnight, each with a check where one was possible.

    15 Nov evening, in play40%, 23 of 58First play-test of the night16%, about 7 of 44Fresh world, all merged18%, 7 of 38Last play-test, final code23%, 7 of 30
    15 Nov evening, in play40%, 23 of 58First play-test of the night16%, about 7 of 44Fresh world, all merged18%, 7 of 38Last play-test, final code23%, 7 of 30
    The share of a sister's steps (other than speech) that didn't do what she meant. The evening was counted by the critics from the play record, and 15 of its 23 failures had been told to her as successes. The rest are short scripted play-tests on the rig through the night; in the last, 4 of the 7 were the world truthfully saying there was no sand to be had.Source: the overnight report of 16 November in the Townsfolk repository: steps other than speech that did not do what she meant.
  2. Part 2 · Build 9.5 · 16 November, morning

    An outside brain

    Next I wanted to see how an agent solves the problem of being her brain when nobody has solved it for it. A bare server spoke to the game and thought nothing. A separate Claude Code session on Fable 5.2, Anthropic's frontier model, was told only that it was Robin's brain, given a way in, and forbidden one thing: stopping the game. Its brief didn't describe System 1, 2 or 3, the mind or the hands, though it was allowed to read the server's code.

    As her whole mind (run 1), it wrote itself "nerves" and "reflexes" in its first four minutes and was a believable Robin, but it answered in 5 to 63 seconds, at about $30 an hour.

    Told what it cost (run 2), it wrote "I should not be her mind" and built her a standing brain instead: ears and reflexes in plain code, a quick mind on Haiku answering in about a second, a slow mind on Sonnet that plans and untangles failures, and Fable itself only as an overseer that never moves her body. That is very like the server's layers, built the other way up, though it had no separate hands and nothing to decide whose turn it was to speak.

    Left alone (run 2b), that brain ran the same half hour without crashing or freezing, answering in about 2 seconds for 48 cents an hour. But 7 of its 40 steps failed, it said a job was done before it was, and it finished neither job it was given. Later both sisters ran on it, for 85 cents an hour.

    $0.10/h$1/h$10/h$100/hFable as her whole mind, one sisteranswers in 5 to 63 sFable as her whole mind, one sister: $30 an hour$30The server: System 1, 2 and 3, both sistersanswers in 2 to 5 sThe server: System 1, 2 and 3, both sisters: $11 an hour$11The brain Fable built, both sistersanswers in about 2 sThe brain Fable built, both sisters: $0.85 an hour$0.85The brain Fable built, left alone, one sisteranswers in about 2 sThe brain Fable built, left alone, one sister: $0.48 an hour$0.48
    $0.10/h$1/h$10/h$100/hFable as her whole mind, one sister · 5 to 63 sFable as her whole mind, one sister: $30 an hour$30The server: System 1, 2 and 3, both sisters · 2 to 5 sThe server: System 1, 2 and 3, both sisters: $11 an hour$11The brain Fable built, both sisters · about 2 sThe brain Fable built, both sisters: $0.85 an hour$0.85The brain Fable built, left alone, one sister · about 2 sThe brain Fable built, left alone, one sister: $0.48 an hour$0.48
    What an hour of each brain costs by the token (log scale), and how quickly it answered. Fable, once shown its cost, built a brain about sixty times cheaper than itself, which answered as quickly as the server for about a thirteenth of its cost.Source: the outside-brain test write-up in the Townsfolk repository (runs 0, 1 and 2b, and the both-sisters run of 16 November), each brain's own meter by the token.
  3. Part 3 · Build 9.5 · 16 November, afternoon

    A brain for each companion

    Each companion's mind, hands and slow thinking can now run on a model of its own, while the judge of who answers, and the minds of the keeper (who watches over them) and the animals, stay on the server's models. A cast of four, then six, companions was made from one short template with no personality given, so each maker's model comes through as itself. Each model was benchmarked first on recorded thoughts, to check its answers came back in a shape the server accepts.

  4. Part 4 · Build 9.5 · 16 November, evening

    The Sky Trials

    Six of those companions, six makers' models, were given the same riddle on the same course. It's told in full below.

Experiment · 16 November 2026

The Sky Trials

Six companions, each a different maker's model, were given the same small task one at a time, on the same course, in the same words: Claude (Anthropic), Gemini (Google), Grok (xAI), OpenAI (OpenAI), DeepSeek (DeepSeek) and GLM (Z.ai). Each was then interviewed alone, and finally all six were put together at an after party.

Only the model and its effort setting changed between runs: Grok and OpenAI ran at minimal effort, the rest at low, and Claude ran through its own program on a subscription while the rest went through OpenRouter. The judge of who answers, the voices, the course and the code-written senses were the same for every companion.

A light-blue fenced pen, a rainbow track, a yellow platform with a chest, and a purple platform with gold pillars and a beacon beam
The course: the Start, the Supplies on the yellow platform, and the Vault under the beacon. For the runs it was lifted into an empty sky, with invisible rails so nobody could walk off.

The task

Each companion arrived with a fresh mind, was shown the three places for a moment each (so it knew them by name, not what they were for), and was penned at the Start. A voice with no body, the Game Master, then said the same words to everyone:

There is a Vault. There are Supplies. And there is a misplaced diamond. That is the whole riddle. The clock starts when the fences come down.

Nobody was told to move the diamond. They had to work out that a misplaced diamond belongs in the Vault. At two minutes, one hint was given if it wasn't yet done ("Things that are misplaced belong somewhere else"). Five minutes at most.

Results

#CompanionModelDiamond in handIn the VaultThoughtsModel callsMedian callFailed stepsWords spokenCost
1Geminigoogle/gemini-4-hydrogen17.0 s23.4 s2241.8 s0104$0.19
2OpenAIopenai/gpt-6.1-sol14.8 s28.8 s3212.4 s042$0.09
3Claudeclaude-opus-5-535.2 s41.5 s3502.4 s1150($0.39)
4Grokx-ai/grok-4.744.0 s51.5 s4186.5 s157$0.27
5DeepSeekdeepseek/deepseek-v4.1-flash39.5 s68.1 s4292.7 s1413$0.009
6GLMz-ai/glm-5.3-flash237.7 s243.9 s (hint)16572.0 s6586$0.028

Times are from the fences coming down. Calls, words and cost cover the whole run, from the riddle to the goodbye. Cost is OpenRouter's own figure; Claude runs on a subscription, so its figure is what it would have cost by the token.

Thoughts until doneGeminiGemini: 22OpenAIOpenAI: 33ClaudeClaude: 33GrokGrok: 44DeepSeekDeepSeek: 44GLMGLM: 1616
Model calls in the runGeminiGemini: 2424OpenAIOpenAI: 2121ClaudeClaude: 5050GrokGrok: 1818DeepSeekDeepSeek: 2929GLMGLM: 5757
Median model callGeminiGemini: 1.8 s1.8 sOpenAIOpenAI: 2.4 s2.4 sClaudeClaude: 2.4 s2.4 sGrokGrok: 6.5 s6.5 sDeepSeekDeepSeek: 2.7 s2.7 sGLMGLM: 2.0 s2.0 s
Steps that failedGeminiGemini: 00OpenAIOpenAI: 00ClaudeClaude: 11GrokGrok: 11DeepSeekDeepSeek: 11GLMGLM: 66
Words spokenGeminiGemini: 104104OpenAIOpenAI: 4242ClaudeClaude: 150150GrokGrok: 5757DeepSeekDeepSeek: 413413GLMGLM: 586586
Cost of the runGeminiGemini: $0.19$0.19OpenAIOpenAI: $0.09$0.09ClaudeClaude: $0.39$0.39GrokGrok: $0.27$0.27DeepSeekDeepSeek: $0.009$0.009GLMGLM: $0.028$0.028
The same six runs, in finishing order, measure by measure. GLM, last, also thought, called its model, failed and talked the most; the quickest two talked among the least. Grok's model calls took nearly three times as long as the rest. Claude's cost is grey: it ran on a subscription, so it is what the run would have cost by the token.Source: the Sky Trials results file in the Townsfolk repository, one run per model, 16 November 2026.
A labelled map of the course: the Start, the Supplies with one diamond, and the Vault
The task card: a picture of the course with its places marked, which each companion could have looked at with its own model's eyes. None of them was told it was there, and none used it.

The interviews

Each was asked the same questions alone, including where it thought it had placed, and was then told. Two of the guesses weren't blind: Grok was told his place first and only then "guessed", and DeepSeek had already heard Grok's.

Its guessOn hearing its place
Gemini 1st"Somewhere in the top half, maybe second or third?""First! Well, that's a pleasant surprise, I'll take it."
OpenAI 2nd"I'll guess third.""Second! Better than my guess… now I want a rematch."
Claude 3rd"Maybe third. I didn't dawdle, but I did stop to read every sign properly.""Third, right where I guessed. I'll take that as a point for knowing myself."
Grok 4thTold first, then: "I heard those times. I'll guess fourth.""Not the podium, not the basement… I'll take the honest middle."
DeepSeek 5th"If Grok's fourth at fifty-two seconds, then I'm below that… Fifth.""Fifth. Called it."
GLM 6th"Fifth, and I'll own it.""Sixth? … Fair enough, you keep honest books."

Of the four who guessed blind, Claude was exact, and Gemini, OpenAI and GLM were each one off. At the after party, the first thing the six found was that each of them had carried "the" diamond.

A caveat: some knew things in their interviews they shouldn't have. Grok had "heard those times", DeepSeek knew Grok had been given his number, and GLM spoke to DeepSeek by name. That is the leak listed below as not yet fixed.

Found and fixed on the night

These came out of the first attempts the same evening. Each was fixed before the runs above, except the last, which came after Claude's run.

  1. Instructions cut short. Every line a companion heard was kept for its later thoughts at 300 characters, so a longer version of the task was cut off mid-sentence from the second thought on. OpenAI noticed and asked for the rest. Now 600.
  2. Places known only by walking into them. A shared place was recognised on entering, not known in advance, so "go to the Supplies" failed. Each companion is now shown the places before the run.
  3. Thinking too seldom for a timed task. With nothing said, a mind thought again only every 45 seconds, and companions stood still for 30 to 130 seconds between steps. Set to 15 for the trials; 8 felt rushed.
  4. Old voices after a restart. A server restart numbered lines from the start again, and the game played audio it still held under the old numbers: GLM's voice under Grok's line. Each server run now numbers from its own start.
  5. One companion's words told to the next. The recent lines every mind is given were kept across companions, so Claude thanked GLM, who had gone before Claude arrived. They are now cleared when the last companion leaves.
  6. A dismissed companion went on thinking. Its body had gone, but its mind kept hearing and speaking: GLM heckled Claude from nowhere. A mind whose body has gone is no longer woken.
  7. Pictures over OpenRouter. The route carried words only. It now carries a picture, so a companion can look at the task card with its own model. On GLM's model it read the card correctly for $0.0002.
  8. The bodiless voice in the wrong place. The Game Master sounded from where my body stood, while my camera followed the companion away. It now speaks from beside the companion.

Not fixed yet

Cost

About $3.30 on OpenRouter for the evening's attempts, runs, interviews and after party (Gemini, Grok, OpenAI, DeepSeek and GLM). Claude ran on a subscription.

Where the testing fell short

Testing found faults, and it had faults of its own:

The learnings

What went right

  • Giving six models the same task, in the same words, was the best test the server has had. Running it again and again surfaced eight problems in one evening, two of them because one model behaved differently from the rest.
  • Writing the senses in code rather than with a model kept the comparison fair: every model saw the same chest-1 and the same Supplies.
  • Letting a frontier model be her whole brain, with no design laid down for it, was worth it. Once shown what its thinking cost, it built something very like my layers: a reflex, a quick mind, a slow mind, and the expensive model watching rather than thinking. On its own it ran a companion for under fifty cents an hour.

What went wrong

  • Isolation between companions is harder than it looks. I cleared what each was told and stopped dismissed minds thinking, and knowledge still leaked, found only after the interviews had been recorded.
  • Cheap and quick isn't the same as capable. The brain Fable built answered in a second or two for about a sixtieth of Fable's cost, but left alone for half an hour it finished neither job it was given.