mlx-optiq
← Models

gemma-4-12B-it-qat-OptiQ-4bit

google/gemma-4-12B-it-qat-q4_0-unquantized, quantized with OptiQ by npario for Apple Silicon; takes images.

communityvision
On disk9.0 GB
Memoryabout 12 GB
Capability Score69.64
Imagesyes

Capability Score breakdown

The six metrics behind this model's Capability Score of 69.64, as reported in the model card. Values are compared with uniform 4-bit (QAT base).

  • This model
  • Uniform 4-bit (QAT base)
Capability Score breakdown: gemma-4-12B-it-qat-OptiQ-4bitMMLU: This model 52.5, Uniform 4-bit (QAT base) 50.9; GSM8K: This model 93.3, Uniform 4-bit (QAT base) 93.1; IFEval: This model 73.6, Uniform 4-bit (QAT base) 72.3; BFCL-V3: This model 72.0, Uniform 4-bit (QAT base) 72.5; HumanEval: This model 91.5, Uniform 4-bit (QAT base) 90.9; HashHop: This model 35.0, Uniform 4-bit (QAT base) 30.0MMLU52.550.9GSM8K93.393.1IFEval73.672.3BFCL-V372.072.5HumanEval91.590.9HashHop35.030.0

Use it

Install

pip install mlx-optiq

Serve it

An OpenAI- and Anthropic-compatible server on this Mac. Every app below can use it.

optiq serve --model npario/gemma-4-12B-it-qat-OptiQ-4bit

OptiQ Code

The terminal coding agent uses whatever model optiq serve is running.

optiq serve --model npario/gemma-4-12B-it-qat-OptiQ-4bit
optiq code

OptiQ Lab

optiq lab --model npario/gemma-4-12B-it-qat-OptiQ-4bit

Claude Code

optiq run claude -m npario/gemma-4-12B-it-qat-OptiQ-4bit

Codex

optiq run codex -m npario/gemma-4-12B-it-qat-OptiQ-4bit

OpenCode

optiq run opencode -m npario/gemma-4-12B-it-qat-OptiQ-4bit

Mistral Vibe

optiq run vibe -m npario/gemma-4-12B-it-qat-OptiQ-4bit

Hermes Agent

optiq run hermes -m npario/gemma-4-12B-it-qat-OptiQ-4bit

OpenClaw

optiq run openclaw -m npario/gemma-4-12B-it-qat-OptiQ-4bit

optiq run starts optiq serve if it is not running, sets the agent up for this session only, and stops the server when the agent exits.

mlx-lm, text only

from mlx_lm import load, generate

model, tokenizer = load("npario/gemma-4-12B-it-qat-OptiQ-4bit")
print(generate(model, tokenizer, prompt="Hello", max_tokens=256))

Details

FamilyGemma-4 · Gemma-4 docs
ArchitectureGemma 4
Base modelgoogle/gemma-4-12B-it-qat-q4_0-unquantized
Size on disk9.0 GB
Memory neededabout 12 GB
Imagesyes, through optiq serve and the Lab
Capability Score69.64 (from the model card)
Downloads185 in the last month
Updated2026-09-15
Hugging Facenpario/gemma-4-12B-it-qat-OptiQ-4bit