The harness matters.

We ran two coding agents through an open harness benchmark. Both drove the exact same local model: a 4-bit Qwen3.6-35B-A3B served by optiq serve. They solve the same tasks. But OptiQ Code moves about five times fewer tokens through the model to get there. On a Mac, every token is your own compute and your own battery. That is the whole game.

There is a claim we have wanted to test properly for a while: that a coding agent built for local models beats a generic one when both drive the same local model. It is easy to assert and hard to prove, because most agent benchmarks measure the model, not the wrapper around it. So we borrowed one that measures the wrapper.

openbench is a from-scratch benchmark for coding-agent harnesses. Its thesis is our thesis: given identical models and tasks, how much does the scaffolding matter? Each task is a self-contained repo with an instruction and a grading script. A task counts as solved only when its checker.sh exits zero, never by the agent's own claim of success. That last part matters. An agent that says "fixed it" and did not gets a zero.

That leaves one variable to study: the agent wrapping the model.

The setup

One model serves every request. We loaded mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit once with optiq serve. It is a 35B sparse MoE with about 3B active per token, quantized to 4-bit, resident on a 36 GB M3 Max. It exposes an OpenAI-compatible endpoint. Both agents talk to that one endpoint, so the model, the quantization, and the KV cache are held fixed. The only variable is the agent driving it.

The two harnesses are OptiQ Code, our terminal agent built for local models, and opencode, a widely-used general coding agent. OptiQ Code plugs into openbench through its bring-your-own-harness manifest, running headless with auto-approve. opencode plugs in through its own adapter, pointed at the same local endpoint.

Measuring tokens the same way

opencode reports its own token counts. OptiQ Code, run as a black-box CLI, does not expose them to openbench. Comparing a harness's self-reported tokens against nothing is not a comparison. So we put a small counting proxy between both agents and optiq serve: it forwards every request untouched and reads the usage field out of each response. Both harnesses are now metered the same way, at the server, using the same definition of a token and a turn.

Both solve the same tasks

Correctness saturates, exactly as openbench's own results predict for capable models. On the six standard tasks, both agents go six for six. On the first hard task, taskflow (a 7,800-line codebase), both solve it outright. The two agents separate only on the second hard task, webcore, where both hit the 600-second cap mid-work. OptiQ Code was nearly twice as far along when time ran out.

If the story ended here, it would be a tie. But once both agents solve the task, the question worth asking is what each one spent to get there.

The cost of a solve

Here is the full run on the six tasks both agents solve. Every number is metered identically through the proxy: prompt tokens sent, output tokens generated, model round-trips, and wall-clock agent time.

Bold is OptiQ Code. Dim is opencode. Both drove the same Qwen3.6-35B-A3B.

Averaged across those six tasks, the gap is not subtle:

Why the leaner agent wins

The token gap comes almost entirely from what each agent sends. opencode carries a large, general system prompt: roughly 7,600 tokens of instructions and tool schemas. It re-sends the growing context on every turn. OptiQ Code is built for one job on one class of model, so its prompts stay lean and it keeps the context it re-sends tight. On the identical task, opencode simply makes the model reprocess far more text on every turn.

On a Mac this is not an accounting detail. There is no per-token invoice, but there is a per-token cost in seconds and in watts. Prompt tokens are the model re-reading context on every step; on local hardware that is real prefill time and real battery. Five times fewer tokens is five times less of the work that makes a laptop warm. It is also why the wall-clock gap is so wide, even though we treat time as the softest number here.

What this means on a Mac

Put the pieces together. A 4-bit 35B model, running entirely on a laptop, solves real coding tasks. That includes a 7,800-line refactor, graded by an external checker, with no cloud and no API key. And the agent you wrap it in decides whether that costs 17k tokens or 83k tokens per task.

For a hosted frontier model, harness overhead is somebody else's server and a line on a bill. For a model on your own machine, it is the machine. The leaner harness finishes sooner, and it leaves more of your context window for the real work. That is the case for a purpose-built local agent, and it is why, on a Mac, we think the harness is the part you should not leave to a general-purpose tool.

On a Mac, for local models, the harness is not a detail. Use one built for the job.

Reproduce it

Everything here runs on hardware you own. Serve the model, then point OptiQ Code at it.

# serve the model once
optiq serve --model mlx-community/Qwen3.6-35B-A3B-OptiQ-4bit --port 8099

# drive it with the local coding agent
export OPTIQ_BASE_URL=http://localhost:8099/v1
optiq code launch .

The openbench manifest, the counting proxy, and the raw per-task results are small enough to rerun on any Apple Silicon Mac with 36 GB. The tie on correctness is the model's; the efficiency is the harness's.