MiniCPM on Apple Silicon

OpenBMB's MiniCPM models, in two lines. MiniCPM-V 4.7 takes images: a 1B, a 35B-A3B MoE that scores 87.61, the highest of any MoE in the lineup, and a 19B REAP of it. MiniCPM5 is text-only and on-device: a 1B and a 2B with a hybrid reasoning mode.

MiniCPM-V 4.7

MiniCPM-V 4.7 is an image-text-to-text model from OpenBMB. The vision tower is a SigLIP NaViT tower with a window merger; image tokens sit on a 2-D canvas with M-RoPE positions. The language tower is Qwen3.5 (dense, 24 layers) in the 1B variant, or Qwen3.5-MoE (40 layers, 256 routed experts, 8 active per token plus a shared expert) in the 35B-A3B. Both use hybrid attention, linear-attention layers with full attention every 4th layer, and a 262k context. This page covers three quants: the 1B at 4-bit, the 35B-A3B at 4-bit, and a 19B REAP of the 35B-A3B with half its experts pruned.

The 1B suits any Mac: 1.9 GB on disk with its vision sidecar. The 35B-A3B fits in memory on a 36 GB Mac; on smaller Macs, --stream-experts on keeps about 4.6 GB resident at the cost of speed. The REAP is 14.6 GB with vision, keeps the same active parameters per token, and gives up knowledge for size: MMLU falls 22 points while code, tool calling and instruction following hold.

The quants

The second number is the vision tower, kept at bf16 in optiq/optiq_vision.safetensors. OptiQ loads it alongside the language tower. Every repo also carries a measured kv_config.json for its full-attention layers.

Capability Score

The standard OptiQ six-metric text eval. The 35B-A3B is second in the lineup only to Qwen3.8-27B (87.98), and ahead of Qwen3.6-35B-A3B (80.03) at the same size.

The REAP keeps 128 of 256 experts in every layer, copied bit-for-bit from the 35B-A3B quant; its KL against the unpruned model is 0.077. How pruning was measured is in What pruning costs.

Images

Serve any of the three and send an image as image_url content:

$ pip install "mlx-optiq>=0.5.25"
$ optiq serve --model mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit

Asked what text is in an image of the words "OPTIQ 42", all three quants answer OPTIQ 42. On a Mac with less than 36 GB, add --stream-experts on to the 35B-A3B.

Hello world

import optiq  # registers minicpmv4_7 and the vision sidecar
from mlx_lm import load, generate

model, tok = load("mlx-community/MiniCPM-V-4.7-1B-OptiQ-4bit")
msgs = [{"role": "user", "content": "Explain why the sky is blue."}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=512))

Stock mlx-lm does not know the minicpmv4_7 architecture, so import optiq comes first. It needs OptiQ 0.5.25 or later.

MiniCPM5

OpenBMB's MiniCPM5 family: Llama-architecture models with hybrid reasoning, tool calling and a 128k context, released under Apache-2.0. The mlx-optiq builds are two HF repos, both sensitivity-aware mixed-precision quants: the 1B fits in 875 MB and is the smallest base in the lineup worth fine-tuning; the 2B fits in 1.8 GB and scores level with the 2.8 GB Qwen3.5-4B quant.

The quant

Per-benchmark breakdown: MiniCPM5-2B

Per-benchmark breakdown: MiniCPM5-1B

Hello world

from mlx_lm import load, generate

model, tok = load("mlx-community/MiniCPM5-1B-OptiQ-4bit")

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Summarize the plot of The Iliad in three sentences."}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
print(generate(model, tok, prompt=prompt, max_tokens=300))

Hybrid reasoning: think or no-think

MiniCPM5's chat template accepts an enable_thinking flag. With it on, the model emits a <think>...</think> block before the answer, useful for math, multi-step planning, or anywhere you want chain-of-thought. The recipe per the model card:

Pass the flag via chat_template_kwargs at the OpenAI endpoint level or as a keyword to apply_chat_template directly. optiq serve forwards chat_template_kwargs verbatim, so you can flip thinking per-request from any OpenAI-compatible client.

Serving

$ optiq serve --model mlx-community/MiniCPM5-1B-OptiQ-4bit --port 8000

# From any OpenAI-compatible client:
$ curl -s http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"mlx-community/MiniCPM5-1B-OptiQ-4bit",
         "messages":[{"role":"user","content":"What is 17 * 23?"}],
         "chat_template_kwargs":{"enable_thinking":true}}'

Fine-tuning

The 1B size makes MiniCPM5 the smallest base in the mlx-optiq lineup that's still capable enough to fine-tune for real tasks. On a 24 GB Mac, LoRA training fits comfortably at max_seq_length=2048 with all 7 Unsloth target modules adapted. No num-layers / sequence-length workarounds needed. The sensitivity-aware LoRA overlay reads the optiq/metadata.json sidecar and gives 8-bit layers 2× the adapter rank of 4-bit layers at the same parameter budget.

$ optiq lora train mlx-community/MiniCPM5-1B-OptiQ-4bit \
      --data ./my_training_data \
      --preset default \
      --max-seq-length 2048