The first version of neal was a plan file and this prompt:

Execute @plans/EMBER_MIGRATION.md. keep going. don't stop unless you are blocked.

I typed that into Codex during a large frontend framework upgrade at work. GPT-5.4 was the current model at the time. I wanted to hand it several months of tedious work, check in periodically, and otherwise let it keep moving. Each time the arrangement broke down, I added something. That eventually became neal, a local CLI that coordinates coding agents around a plan.

The migration

Our frontend codebase had been due for an upgrade from Ember.js v3 to v5 (ahem… v6) for a long time. This would impact virtually all of our code: thousands of files, significant syntax changes, and replacement of deprecated patterns. Most of the edits were repetitive, but not easily scriptable. Some shared components would have to be completely reworked. We estimated the manual work to be over 300 engineering hours. Even with our reasonably comprehensive test suite, the project carried significant risk.

With a lean engineering team, it seemed like an impossible task. Until coding agents came along. In mid-2025, I excitedly put Claude Sonnet 4 on the task, only to watch it play whack-a-mole endlessly with our test suite, fixing and causing the same bugs over and over. The amount of babysitting required at the time was too much to justify the mediocre results. The migration was shelved.

Earlier this year, we felt like it was time to try again. Codex and Claude models had improved significantly at planning, coding, and working autonomously. I hoped that with a detailed migration plan doc, one of the frontier coding agents could work through the migration incrementally on its own, with me checking in to review progress along the way.

Codex kept stopping

The migration plan had links to Ember upgrade resources, commands for running the test suite, and an instruction to work on 10 to 20 related files at a time. It also kept a list of files that still needed to be migrated, so Codex could pick up where it left off when interrupted.

For a while, the bare prompt worked surprisingly well. Codex could work on its own for long stretches. When I restarted it, the plan still showed what was finished, what came next, and which constraints applied to the remaining work.

The problem was keeping it going. Codex would finish a batch and stop for no apparent reason. I would ask, “Why did you stop iterating on the migration?” The reply was often, “I didn’t,” despite having been idle for hours.

Long sessions uncovered a second problem. Codex gradually drifted from the instructions it had followed earlier. I added “Re-read this migration plan before starting on the next batch of files” to the plan, but that wasn’t always enough. I found a name for this behavior: context rot. Over a long session, old instructions and decisions lose influence as the context grows or is compacted.

I tried to solve both problems inside Codex. I turned the prompt into a $work-autonomously skill with a Stop hook, textual markers to signify that a batch of files had been completed or a blocker encountered, and an instruction to reset the agent context via /new before each batch. The skill helped, but Codex still didn’t always remember to start a new context.

I asked Codex:

you don't always follow the instruction to start a new context with every chunk of work. how feasible would it be to build a node app to direct codex to do a chunk of work?

This was the precursor to neal: a node script using the Codex SDK to execute the migration in a loop.

A second model for review

Reviewing the incremental migration commits exposed another issue. Code quality was not always the greatest. I decided to incorporate a common workflow of mine: have Codex implement a thing, have Claude review it, and have Codex respond to Claude’s feedback. I would repeat that until both models agreed, then review the change myself.

I added this to the node script using the Claude SDK to review the latest commit. The script captured Claude’s feedback and sent it to Codex. At this point I decided to give my orchestrator a name: neal.

The name comes from anneal, the process of heating and cooling a material in controlled cycles until its internal stresses settle. I dropped the “an” and kept the process.

What neal does

neal is a local CLI that coordinates planner, coder, and reviewer roles around a plan document. You choose the provider and model for each role. These days I am using Anthropic’s Fable as the coder and OpenAI’s Sol as the reviewer.

neal run is the normal workflow. It starts by sending a rough plan through the planner and reviewer. They give it an execution shape, split larger work into scopes, and fill in the implementation approach, verification, and success conditions. They leave file-level discovery and local implementation choices for the coding phase. You can use neal plan and neal execute separately when you want to read or edit the refined plan before starting the implementation.

During execution, the coder starts each scope with a fresh context, implements and verifies the work, then commits it. The read-only reviewer keeps its context across scopes and checks each commit against that scope. Findings go back to the coder until the reviewer accepts the work.

When a run gets stuck, neal can ask the reviewer model for a bounded consultant turn. This covers a blocked coder, a stalled coder/reviewer loop, or a bad scope split. The consultant can diagnose the problem and give the coder specific direction, but it can’t edit the code or waive verification. If there isn’t a safe way forward, neal asks for human direction. A scope that turns out to be too large can also split into a sub-plan.

After every scope is accepted, the coder and reviewer make one final pass over the complete implementation and plan. Any remaining findings go back through the same implementation and review loop. neal then squashes the scope commits unless you tell it not to. Run state, transcripts, and review artifacts live under .neal/, so an interrupted run can resume from the last recorded phase.

neal has native adapters for Codex and Claude, plus an OpenAI-compatible adapter for services such as OpenRouter and local endpoints such as Ollama or vLLM. The native adapters can use existing Codex and Claude subscription authentication. I built it this way because a long planner/coder/reviewer loop can be much cheaper on subscription plans than paying per token. Also, I am fortunate enough to work for a company that reimburses both my Codex and Claude Max plans ;-)

The migration landed

neal finished the migration. The migration branch landed with 549 commits and touched over 3,000 files. It also added around 13,000 lines of test code. After weeks of cooking in our customers’ acceptance testing environment, we shipped it to production last Sunday.

I would like to be able to say it took a few days, but since I was building out the orchestrator as the migration was ongoing, I don’t really know the true execution time of the project. It took around a month to complete, but most of my work was on neal. I can say that was a lot more fun than the migration itself.

SWE-bench Pro

I was also curious about how a Codex+Claude setup would perform against standard coding benchmarks. After going down a bit of a rabbit hole trying to find recognized benchmarks that still had signal and that could support a custom orchestrator, I eventually settled on SWE-bench Pro.

Starting with 105 cases that Codex (GPT-5.5) was unable to solve on its own, I ran neal with Codex in all three roles. That resulted in 8 cases passing that had previously failed. Then I ran the same cases through neal with Codex planning/coding and Claude (Opus 4.8) reviewing. This resulted in 15 cases passing. I had hoped for better results, but the experiment did show the value of the parts of neal I personally find useful: a planning loop that generates a scoped, reviewed plan; an implementation loop between coder and reviewer; and a different model in the reviewer role. The harness, methodology, and per-case results are in the neal-swebench repo.

One nice by-product of getting so many iterations on neal (all the benchmarking and the Ember migration) was the collection of run artifacts. I had LLM chat transcripts, plan revisions, a record of how many loops were needed in each phase, etc. I was able to periodically (using a coding agent, of course) analyze recent runs for failures and inefficiencies. These findings were fed back through neal to improve its own design.

Model compatibility

A model can answer a prompt and still be unable to run one of neal’s roles. neal expects structured output, tool calls, file inspection, and specific review verdicts. Plenty of otherwise capable models fail one of those protocols.

neal compat runs small, deterministic tasks against the coder, reviewer, and planner roles and reports PASS or FAIL. A pass means only that the model can speak neal’s protocol, not that it’s good at planning, coding, or reviewing.

As part of this project, I ran the compatibility check against 90 OpenRouter models, using Codex with GPT-5.5 as the known-good partner for each candidate. Forty-four passed every role. The dated results and exact failure counts live in the repo’s compatibility list.

neal compat also caught bugs in neal itself. Five flagship models failed in exactly the same way because of an overly strict rule in my validator. Another group of reviewer failures came from Ubuntu 24.04 AppArmor blocking Codex’s read-only sandbox. In both cases, the harness was wrong rather than the models.

Try it

npm install -g @navels/neal

The source and docs are on GitHub. If you find it useful, please consider donating to MusiCares. It’s a non-profit giving musicians a place to turn to in times of financial, personal, or medical crisis. You can also find me on LinkedIn.