mlx-optiq

Changelog

Every version of mlx-optiq published to PyPI, what shipped, and why. The source is the repo's CHANGELOG.md; this page is its public mirror.

v0.5.8

Added

  • optiq serve --ngram-draft 8: prompt-lookup speculative decoding for any model. It drafts tokens copied from the conversation, verifies them in one pass, and picks the draft length per step from how often drafts are accepted; 1.3-1.5x on real coding-agent turns, Qwen3.5/3.6 hybrids included.

Fixed

  • optiq kv-cache measured sensitivity on bfloat16 logits at one position, where rounding noise was larger than the signal: layers were ranked on noise, and some models measured exactly zero and were refused. It now measures in float32 over every position.
  • optiq kv-cache on a model with no quantizable KV cache (latent attention) reported a broken measurement instead of saying the architecture is unsupported.

v0.5.7

Added

  • optiq code import <trace.jsonl> — continue a saved trace as a session, so a run that died to an outage resumes instead of starting over.
  • task_list — a plan the harness corroborates: each finished item records whether anything was actually run.
  • finish_gate (headless, off by default) — refuses a finish while the tree does not build.
  • request_extra — a JSON object merged into every chat request, for provider pinning and routing.
  • max_boosts and boost_cooldown settings.
  • optiq code -c -p GOAL continues the previous conversation headlessly.

Changed

  • Requests stay inside the model's context window: the output request shrinks first, then compaction thins old context, and a run that still cannot fit ends cleanly with its patch saved.
  • After-edit verification speaks the repository's language — per-file syntax checks for nine languages, plus a project build run in the nearest directory above the edited file that holds a marker.
  • The stall ladder escalates on turns spent working outside the repository, says each thing once, and never ends a run with nothing to hand back.
  • The completion check also runs when the agent finishes by calling done.
  • Nothing is tested at session start, and a failing test is labelled pre-existing or new.
  • Long tool results keep their head and their tail instead of losing the summary line.
  • The TUI leaves the conversation in your scrollback, releases the mouse so select-and-copy works, and queues a task typed mid-run.

Fixed

  • MTP ignored the prompt cache and re-prefilled the whole conversation on every request, a bug present since May.
  • optiq serve: a dead generation thread reported healthy, a request without max_tokens was cut at 512, and a model's top_k: -1 failed every request.
  • Responses API: blank user turns before every request, broken replayed histories, dropped prior-turn reasoning, and missing cached-token counts.
  • A run that committed its own work, or probed outside the repository, reported no patch.
  • A dropped chat stream ended the run instead of being retried.
  • optiq kv-cache calibrated on one sentence repeated 25 times and said nothing about it.
  • The MTP head of an expert-pruned quant was built with the parent's expert count.
  • gofmt -l reported an unformatted Go file as a syntax error.

Removed

  • /yank — use /copy.

v0.5.6

Changed

  • auto_verify replaces the after-every-edit full test run. The harness used to run the entire suite after any turn that changed code — 26 full runs of a 10,093-test repo in a single task, which was most of the wall clock. It now parses the edited files instead: that catches the failure that matters (an edit leaving a file unimportable, which once broke 3,335 tests and stayed invisible for 14 edits) in milliseconds. Set auto_verify to full for the old behaviour or off to leave testing entirely to the model. Measured on the same task, model and endpoint: 116 min → 50 min, same 103/103 pass.
  • run_tests takes a target. It previously took no arguments, so the only test the model could run was all of them. Pass a path, a directory, or k:<expr> for a -k expression. With the full-suite crutch gone the agent began testing on its own — 0 self-initiated runs before, 15 after.

Added

  • Traces carry model_seconds, turn_latency_p50 and turn_latency_max. A run that takes several times another harness's on the same endpoint is either the model or the harness, and nothing in the trace could tell those apart.

v0.5.5

Removed

  • replace_lines is gone. Line numbers go stale the moment anything above them shifts, and the edit still applies cleanly — so a wrong edit reported success. On a benchmark task it spliced text into the wrong place and left a file with an IndentationError, which failed 3,335 previously-passing tests. Neither Claude Code nor OpenCode has a line-addressed editor. edit_file's uniqueness-gated match ladder covers the same ground and fails loudly instead. Repeated edit failures now send the agent back to read_file rather than to a line range.

Changed

  • git is read-only. It ran any subcommand and is not gated, and it is mounted in the read-only plan preset — so git reset --hard, git checkout ., git clean -fdx and git push were reachable with no approval prompt, in a mode documented as unable to change anything. Reads (status, diff, log, show, blame, …) still work; writes must go through bash, which is gated.
  • edit_file and write_file require the file to have been read this session, as Claude Code and OpenCode both enforce. An anchor is a claim about what the file holds now. Creating a new file needs no read, and a file the agent wrote counts as read.
  • bash takes workdir and timeout. Each call is a fresh shell, so cd never persisted; workdir is the supported way to run elsewhere.

v0.5.4

Fixed

  • optiq code export dropped the token accounting. A trace recorded how many turns a task took but not what they spent, so two harnesses running the same model could be compared on pass rate and nothing else. The header now carries prompt_tokens, completion_tokens, total_tokens, cached_tokens, reasoning_tokens, cost_usd and priced.

v0.5.3

Added

  • disable_tools withholds named tools from a run, e.g. OPTIQ_CODE_DISABLE_TOOLS=web_search,web_fetch. They are dropped from what the model is offered and refused at dispatch, so a model that names one anyway is told it is off. For air-gapped runs, and for benchmarks where reaching the open web lets the agent look up the fix instead of deriving it.

Fixed

  • optiq code config and optiq config printed api keys in full. They are now masked to a prefix, a suffix and a length — enough to tell which key is in place, not enough to use.
  • trace_tags is now a real setting (OPTIQ_CODE_TRACE_TAGS, with OPTIQ_TRACE_TAGS still honoured). It was read straight from the environment, so it never appeared in optiq code config and could not be set from a config file.

Earlier releases

The 14 releases before v0.5.3. Click any version to expand it. Everything before v0.4.24 is in the archive.

v0.5.2

Fixed

  • optiq code could not run at all on a clean pip install mlx-optiq: it failed with "MCP servers are configured but the 'mcp' package is not installed" on machines that had no MCP servers configured. The optional import now happens only when a server is actually enabled, and the message names which ones want it.
v0.5.1

Fixed

  • Boost from OpenCode never fired: it sends the prompt JSON-quoted, so the trigger never matched and the local model answered every time, with no error.
  • Boost from OpenClaw never fired either: it stamps the time onto every message, so the trigger no longer started the line. Any bracketed tag a client adds is now allowed in front of it.
  • One /boost could cost two credits: clients send a separate title-generation call carrying the same text, and it opened its own episode. Affected OpenCode and Claude Code.
  • Boost failed outright for clients with rich tool schemas. The cloud model's tool API takes an OpenAPI subset, not JSON Schema, and rejected the whole request over const, patternProperties, exclusiveMinimum and uniqueItems. Schemas are now rewritten into the accepted subset.
  • optiq cloud status and every boosted reply named the backing frontier model. They now report optiq-boost.
  • optiq code told a stalled turn to edit files even when the goal was a question, so asking one could produce edits you never requested. It now offers answering too, and never demands a write in plan mode.
  • Site: the favicon was still the old one (cached at an unchanged URL), and /favicon.ico did not exist.
v0.5.0

Added

  • OptiQ Cloud Boost. /boost (or boost: for clients that eat slash commands) hands one hard turn to a frontier model, then gives control straight back. It lives in optiq serve, so Claude Code, Codex, OpenCode, OpenClaw, Hermes Agent, Mistral Vibe, the Lab and OptiQ Code all get it with no plugin.
  • One Boost is one episode, not one API call: a whole tool loop costs one credit.
  • Works on all three surfaces optiq serve exposes, streaming or not: chat/completions, Anthropic messages, and Responses (including previous_response_id chains).
  • optiq cloud login | status | logout. The token is minted when the CLI collects it, so none is ever stored server-side; logout revokes rather than forgets.
  • OptiQ Code: /boost [task], a bare /boost to retry the last request, and manual / suggest / auto modes. The Lab gets a per-message Boost toggle.
  • 20 free Boosts on sign-up; 200 for $20 as a one-off pack.
  • New pages: OptiQ Cloud and its docs.

Changed

  • New design across the site, the Cloud dashboard and OptiQ Lab. Lighthouse 100 on desktop.
  • optiq serve always sends a content key, null when the model produced only reasoning. Clients reading message["content"] used to hit a KeyError.
  • Boost usage follows OpenAI's convention: reasoning inside completion_tokens, cached inside prompt_tokens. Clients were undercounting a Boost roughly tenfold.
  • optiq code reports cost in Boosts, not dollars: local + 1 Boost · 14 left.

Fixed

  • A failed Boost explains itself and says whether you were charged, instead of printing a Python errno.
  • Benchmark eval tests skip without the datasets extra instead of failing.
v0.4.34

Changed

  • HashHop's sample generator is vendored (MIT, from upstream magicproduct/hash-hop), so no benchmark-specific package has to be installed for optiq eval. It is 4.6 KB importing only stdlib and numpy, byte-identical to the generator every published HashHop score was produced with, and a test asserts that against upstream. Removes two traps: the package is not on PyPI so it could only be declared as a git URL, which PyPI rejects — that made 0.4.33 fail to upload — and installing the fork instead of upstream downgrades mlx-lm and silently removes model families.
v0.4.33

Fixed

  • Image input was broken on mlx 0.32.2 for every Qwen3.5/3.6 vision model — the vendored tower passed an array where mx.repeat now requires an int.
  • mlx-community/gpt-oss-20b-OptiQ-4bit could not load: its per-layer quantization entries named no mode, so they inherited the base's top-level mxfp4 while OptiQ had quantized them affine, and mlx raised Scale type must be uint8 on the first embedding lookup. Weights were always correct; the config is republished. The converter now names the mode per layer, and the release contract checks the declared mode against the scale dtypes in the safetensors header.

Added

  • optiq prune-experts removes low-value routed experts from a quantized MoE, implementing REAP (Cerebras, ICLR 2026). Retained experts are copied bit-for-bit; active parameters per token are unchanged. Qwen3.6-35B-A3B at 50% retention: 22.1 → 13.9 GB, 24.5 → 11.6 GB peak, Capability Score 80.03 → 76.57, the loss almost entirely MMLU. Gemma-4-26B-A4B: 75.76 → 68.13.
  • 10 MoE architectures supported. 10 REAP variants published to mlx-community. Diffusion LMs are refused.
  • The command reports KL against the unpruned model before writing and warns above 1.0. Eight candidates were rejected: six on that measure plus a generation check against the parent, and both Nemotron-H 30B-A3Bs on a full Capability Score — they pass KL and a smoke test but lose ~15 points, with GSM8K and HumanEval down more than 20 each.
  • The ranking rule is measured per checkpoint, not assumed, and only overrides the family default by a clear margin.
  • Profiling streams experts from SSD when the checkpoint will not fit: Qwen3.5-122B-A10B (43 GB) profiles in ~6 GB.
  • Grouped routers get a group-balanced plan — groups are positional, so a global top-N renumbers experts into groups they were never trained in.
  • New docs page: Expert pruning.
  • End-to-end tests that serve a real model and exercise text, image input, tool calling and optiq code over HTTP.

Changed

  • optiq eval --task all no longer runs KL by default; pass --with-kl. It is the only step that holds a second model in memory and is not part of the Capability Score. --skip-kl is accepted and ignored.
  • The KL reference comes from the model's own optiq_metadata.json: a quant against its base, a pruned model against the quant it was pruned from.
  • Expert profiling hooks the router and switch submodules instead of copying each MoE block's forward, so it leaves the model's output bit-identical and a new architecture is a table entry.
  • Pruned models rewrite optiq_metadata.json with the prune record instead of inheriting the parent's.
  • hashhop is a declared dev dependency, from upstream. The fork's pins downgrade mlx-lm and silently remove model families.
  • Docs, CLI reference and site copy cover pruning. The CLI reference listed seven commands; there are twelve.
  • The homepage's structured-data version had been stuck at 0.4.7 for 26 releases. The SEO audit now checks it against pyproject.
v0.4.32

Fixed

  • optiq code -p raised ImportError on every run — one relative import was a level too high. Broken since 2026-08-14.
  • /code, /cli and /lab showed a stale version.
  • The lab_server test fixture allowed 30s for a boot that takes 37s on slower machines.

Added

  • A headless test that runs the real CLI, and static checks that every relative import resolves to a module and symbol that exist.
v0.4.31

Fixed

  • The third-party MCP tests never ran in a full test run. They are sync tests calling asyncio.run, and the Lab suite runs before them and leaves an event loop behind, so all ten failed whenever the whole gate ran and passed only in isolation. These are the only tests that exercise real third-party MCP servers.
  • MCP tools were all treated as write tools on mcp 2.x. The SDK renamed the read-only annotation, OptiQ read only the old name, and every MCP tool came through as not-read-only: each prompted for approval, and plan mode hid all of them. Both spellings are now accepted.

Changed

  • mcp is pinned to >=1.23.3,<2, in a new [mcp] extra. It was previously undeclared, so the SDK you got depended on when your environment was built. Measured range: 1.23.3 through 1.29.1 pass, 2.0.0 and 2.1.1 do not.

Added

  • MCP tests run against both SDK generations, and the adversarial server no longer depends on the SDK to send a malformed tool listing.
  • Packaging guards for the mcp pin: declared, bounded, and obeyed by the installed version.
v0.4.30

Fixed

  • MiniCPM5 tool calls were never executed. Its chat template teaches <function name="f"><param name="p">v</param></function>, and nothing parsed it, so the model emitted well-formed calls as ordinary content and every client executed nothing. Fixed on both paths: client-side parsing, and a parser registered with mlx-lm so optiq serve returns structured tool_calls. Handles CDATA values, several calls per reply, and types arguments from the tool schema (the format carries no type marker of its own).
v0.4.29

Changed

  • optiq serve refuses to start on a model it cannot load, exiting non-zero instead of serving. mlx-lm's /health is hardcoded to 200 and never checks model state, so a directory with no weights produced a server that answered /health OK and then hung every request (measured: HTTP 000 after 60s). Anything supervising it saw a healthy process. Local-only check: a repo id that is not cached yet is still downloaded as before.
v0.4.28

Fixed

  • DeepSeek-V4 tool calls still did not work when served. 0.4.27 registered a parser but nothing reached it: optiq serve matches tool_call_start as a token sequence, and > merges with the following newline into one token, so the marker our own template emits never matched. The marker is now bracket-free.
  • Tools were dropped for any client that sent no system message. The template rendered its tool instructions only inside the system turn, so a plain OpenAI client got a 64-character prompt and a model that answered in prose. chat_template.jinja is also re-uploaded to the published quant, since it is copied into the model repo at conversion time.
v0.4.27

Fixed

  • DeepSeek-V4 tool calls were never executed. V4 ships no chat template, so OptiQ supplies one that teaches the model the DSML format (<|DSML|invoke name="...">), and nothing could parse it back. Handles multiple invokes per block, string="false" JSON values, and blocks truncated at the token cap.
  • DSML tool calls are parsed server-side too. mlx-lm picks a tool parser by pattern-matching the chat template and has no DeepSeek entry, so optiq serve returned the block as plain content: third-party clients saw prose and --task bfcl --served would have scored zero.
  • The Lab never compacted a long conversation. It grew until --max-context capped it, and that cap is a RotatingKVCache, which evicts from the front: system prompt first, mid-message, silently. The Lab now compacts at 80% of the server's window, archives what it drops, and emits a compacted event.

Added

  • A contract test round-trips every chat template OptiQ ships against the tool-call parser. Any format we teach a model must be one we can read back.

Changed

  • Compaction moved to one implementation shared by OptiQ Code and the Lab.
v0.4.26

Added

  • optiq eval --served URL scores a model through a running optiq serve instead of loading it into the eval process. Served BFCL reads the tool_calls the server returns rather than re-parsing text.

Fixed

  • Reasoning models scored 0 when served. mlx-lm splits output into message.reasoning and message.content and omits either when empty. Both are rejoined now; reasoning_content accepted as an alias.

Changed

  • All six benchmarks generate through one backend, so prompts, greedy decode and the per-arch repetition penalty are shared by both paths.
  • --served refuses --kv-bits, --kv-config and --stream-experts* (they patch this process, not the server), and --task kl, --task smoketest and MMLU without --reasoning (all need raw logits).
v0.4.25

Fixed

  • Gemma-4 with a per-layer kv_config no longer dies past ~10k tokens. Verified on gemma-4-26B-A4B-it-OptiQ-4bit to 14169 tokens and pinned by a regression test.
  • Only the first of several chained Devstral tool calls was executed. A turn asking to read two files ran one and dropped the other.
  • The auto context cap could size a KV window Metal refuses to allocate. It is now bounded by the device's recommended working set as well as free RAM (Devstral-24B on an M3 Max: 48128 to 33792 tokens).
v0.4.24

Fixed

  • Gemma-4 with quantized KV crashed once anything else had touched attention in the same process. The dispatch was installed on sys.modules, which is not always the dict the model's classes execute in. It now rebinds through the live model and re-asserts per request.
  • Flash-attention training reverted the KV dispatch on exit and the installer latched, so training then serving in one process left Gemma-4 quantized KV broken for the life of it.
  • LoRA training could write an all-NaN adapter, then offer it for mounting. A batch with no unmasked target tokens made the loss compute 0/0 and the global grad-norm clip spread it everywhere. The denominator is clamped and non-finite adapters are refused.
  • Devstral's tool calls were dropped when served. The Mistral parser required a [TOOL_CALLS] marker that mlx-lm's server had already consumed. It is optional now.
  • LoRA trained the base model. apply_sensitivity_aware_lora never froze it, so 133 non-quantized tensors on Qwen3.5-0.8B were updated at the LoRA learning rate and written into the adapter.