mlx-optiq
← Models

gemma-4-12B-it-qat-OptiQ-4bit-MTP

mlx-community/gemma-4-12B-it-qat-OptiQ-4bit, quantized with OptiQ by rariruluis for Apple Silicon; takes images.

communityvision
On disk9.8 GB
Memoryabout 13 GB
Parent score69.64 gemma-4-12B-it-qat-OptiQ-4bit
Imagesyes

This model reuses the per-layer bit allocation of gemma-4-12B-it-qat-OptiQ-4bit, whose measured Capability Score is 69.64.

The parent's Capability Score breakdown

This model has no Capability Score of its own. The chart shows the six metrics behind its parent gemma-4-12B-it-qat-OptiQ-4bit's score of 69.64, taken from the parent's model card and compared with uniform 4-bit (QAT base).

  • gemma-4-12B-it-qat-OptiQ-4bit
  • Uniform 4-bit (QAT base)
The parent's Capability Score breakdown: gemma-4-12B-it-qat-OptiQ-4bitMMLU: gemma-4-12B-it-qat-OptiQ-4bit 52.5, Uniform 4-bit (QAT base) 50.9; GSM8K: gemma-4-12B-it-qat-OptiQ-4bit 93.3, Uniform 4-bit (QAT base) 93.1; IFEval: gemma-4-12B-it-qat-OptiQ-4bit 73.6, Uniform 4-bit (QAT base) 72.3; BFCL-V3: gemma-4-12B-it-qat-OptiQ-4bit 72.0, Uniform 4-bit (QAT base) 72.5; HumanEval: gemma-4-12B-it-qat-OptiQ-4bit 91.5, Uniform 4-bit (QAT base) 90.9; HashHop: gemma-4-12B-it-qat-OptiQ-4bit 35.0, Uniform 4-bit (QAT base) 30.0MMLU52.550.9GSM8K93.393.1IFEval73.672.3BFCL-V372.072.5HumanEval91.590.9HashHop35.030.0

Use it

Install

pip install mlx-optiq

Serve it

An OpenAI- and Anthropic-compatible server on this Mac. Every app below can use it.

optiq serve --model rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

OptiQ Code

The terminal coding agent uses whatever model optiq serve is running.

optiq serve --model rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP
optiq code

OptiQ Lab

optiq lab --model rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

Claude Code

optiq run claude -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

Codex

optiq run codex -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

OpenCode

optiq run opencode -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

Mistral Vibe

optiq run vibe -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

Hermes Agent

optiq run hermes -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

OpenClaw

optiq run openclaw -m rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP

optiq run starts optiq serve if it is not running, sets the agent up for this session only, and stops the server when the agent exits.

mlx-lm, text only

from mlx_lm import load, generate

model, tokenizer = load("rariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP")
print(generate(model, tokenizer, prompt="Hello", max_tokens=256))

Details

FamilyGemma-4 · Gemma-4 docs
ArchitectureGemma 4
Base modelmlx-community/gemma-4-12B-it-qat-OptiQ-4bit
Size on disk9.8 GB
Memory neededabout 13 GB
Imagesyes, through optiq serve and the Lab
Family referenceThis model reuses the per-layer bit allocation of gemma-4-12B-it-qat-OptiQ-4bit, whose measured Capability Score is 69.64.
Downloads268 in the last month
Updated2026-10-01
Hugging Facerariruluis/gemma-4-12B-it-qat-OptiQ-4bit-MTP