Nanbeige on Apple Silicon
Nanbeige4.2 is a looped transformer: a 22-layer decoder stack that runs twice with the same shared weights (num_loops: 2), so the model does the compute of a much deeper network without carrying the extra parameters. Each loop keeps its own KV cache. It is a bilingual English/Chinese reasoning model that opens with a short chain of thought before the answer.
Needs OptiQ 0.4.6+
Stock
mlx-lm has no nanbeige class, so OptiQ ships a vendored, mlx-native port of the looped architecture that registers with mlx-lm on import optiq. This is the same approach used for dhara. Run pip install "mlx-optiq>=0.4.6", then import optiq before mlx_lm.load. Earlier releases cannot load this repo.
The quant
| Model | bf16 size | OptiQ size | Compression | Precision |
|---|---|---|---|---|
| Nanbeige4.2-3B-OptiQ-4bit | 8.3 GB | 3.3 GB | 2.5× | 5.5-bit mixed |
Hello world
hello.pypython
import optiq # registers the nanbeige arch with mlx-lm from mlx_lm import load, generate model, tok = load("mlx-community/Nanbeige4.2-3B-OptiQ-4bit") prompt = tok.apply_chat_template( [{"role": "user", "content": "Explain the looped-transformer idea in two sentences."}], tokenize=False, add_generation_prompt=True, ) print(generate(model, tok, prompt=prompt, max_tokens=300))
How looping works
The 22 physical layers execute as 44 forward steps (two passes), each pass carrying its own KV cache, so a prompt sees the effective depth of a 44-layer network at the parameter and disk cost of a 22-layer one. OptiQ's per-layer sensitivity pass runs over the physical layers; the allocation lands at 5.5-bit mixed precision, 8.3 GB bf16 down to 3.3 GB.