Family guide · Mistral

Mistral on Apple Silicon

Mistral's Devstral is a tool-calling, agentic coding model built on the Mistral-Small-3.1 base, which means it is quietly multimodal: the language tower does the coding work, and a vision encoder rides alongside for image input. The first OptiQ quant of the family is Devstral-Small-2-24B at 4-bit mixed precision. OptiQ quantizes only the language tower; the vision encoder is kept at bf16 in a sidecar (optiq/optiq_vision.safetensors), so the same repo loads text-only under stock mlx-lm and full image+text under optiq serve. It loads through mlx-lm's mistral3 class.

The quants

Model	Size on disk	Capability Score	vs uniform-4	Best for
Devstral-Small-2-24B-Instruct-2512-OptiQ-4bit	15.4 GB	78.60	+1.95	Agentic coding, tool use, image input

Coming, the Small-4 119B flagship Mistral-Small-4-119B is a large sparse-MoE. Its OptiQ quant is an extreme 2-bit static build, sized to run on a 36 GB Mac by streaming its routed experts off SSD, the same approach as the Nemotron-3 Super 120B. No Capability Score on that one by design, an extreme quant ships with a coherence demo instead.

Per-benchmark breakdown, Devstral 24B

Benchmark	uniform-4	OptiQ-4 (mixed)	Δ
MMLU (5-shot, 1000)	79.4%	80.8%	+1.4
GSM8K (3-shot CoT)	88.1%	87.2%	-0.9
IFEval (strict)	68.0%	68.6%	+0.6
BFCL V3 (tool calls)	88.5%	89.5%	+1.0
HumanEval (pass@1)	82.9%	83.5%	+0.6
HashHop (overall)	53.0%	62.0%	+9.0

The win is long-context retrieval Devstral wins five of six benchmarks and clears uniform 4-bit by +1.95 Capability Score. The standout is HashHop, +9 points, where the mixed-precision allocation keeps the multi-hop retrieval that a flat 4-bit quant loses. 179 tensors land at 8-bit, 102 stay at 4-bit, for 5.01 BPW / 15.4 GB. Every metric gets one equal vote; disk size sits next to the score as a second axis. See the eval-framework writeup.

Hello world

hello.pypython

from mlx_lm import load, generate

model, tok = load("mlx-community/Devstral-Small-2-24B-Instruct-2512-OptiQ-4bit")

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Write a Python function that returns the nth Fibonacci number."}],
    tokenize=False,
    add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=300))

Tool calling

Devstral emits a tool call as [TOOL_CALLS]name[ARGS]{...} with no closing marker, the call is terminated by EOS. Stock servers read that EOS as a stop and drop the buffered call, so the agent takes a single action and halts. optiq serve redirects the tool state's stop edge so the call is captured and the turn continues, and it normalizes tool-call arguments before chat templating. Multi-step agent loops work out of the box.

terminalbash

$ optiq serve --model mlx-community/Devstral-Small-2-24B-Instruct-2512-OptiQ-4bit --port 8000

# OpenAI + Anthropic-compatible; tool calls and image_url both supported.

Image input

Because the base is Mistral-Small-3.1, the vision tower travels with the quant in optiq/optiq_vision.safetensors at bf16. The vision path runs only when a request actually carries an image, text-only requests never touch the sidecar. Send an image_url content part through optiq serve or the Lab; see the vision guide.