Same model, four ways to make it 4-bit.
Three repositories on Hugging Face put 4bit in the name of the same Gemma-4-12B. They run from 6.3 GB to 10.2 GB, and no two of them quantize it the same way. This page puts the six-benchmark Capability Score next to the download size for each, so you can see what the extra gigabytes buy.
Spending bits where the damage is.
Both quants below start from google/gemma-4-12B-it. One assigns every layer 4 bits. The other measures each layer's sensitivity first and raises the ones that cannot take it.
| Benchmark | OptiQ mixed 4/8 | Uniform 4-bit | Delta |
|---|---|---|---|
| MMLU 5-shot, 1000 | 42.6% | 34.4% | +8.3 |
| GSM8K 1000, 3-shot CoT | 93.4% | 90.1% | +3.3 |
| IFEval full set, strict | 73.9% | 71.2% | +2.8 |
| BFCL-V3 simple 200 calls | 71.0% | 71.5% | −0.5 |
| HumanEval 164, pass@1 | 88.4% | 76.8% | +11.6 |
| HashHop long-context retrieval | 40.0% | 27.0% | +13.0 |
| Capability Score | 68.23 | 61.83 | +6.40 |
| On-disk size | 8.3 GB | 6.3 GB | +2.0 |
The Capability Score is the mean of the six, unweighted. Size sits next to it rather than inside it.
HashHop and HumanEval move the most. Both lean on a handful of layers that a flat quant crushes along with everything else, which is what the sweep picks up. BFCL goes the other way by half a point. At 200 calls that sits inside the confidence interval, so call it level. It stays in the table because dropping it would make the other five look better than they are.
The 2.0 GB is real. Raising 157 of 329 components to 8 bits costs what it costs, and on a 16 GB Mac that headroom may not be there. Check the size before the score.
Three repositories, one name, three different quants.
Each repository's config.json carries a quantization block. Read the three side by side and the shared 4bit suffix covers three different recipes.
| Repository | Size | What it actually does |
|---|---|---|
gemma-4-12B-it-4bit | 6.3 GB | Uniform 4-bit. No per-layer overrides at all. |
gemma-4-12B-it-OptiQ-4bit | 8.3 GB | Mixed. 172 components at 4 bits, 157 at 8, chosen by measured sensitivity. |
gemma-4-12B-it-qat-4bit | 10.2 GB | Mixed. 144 components pinned at 8 bits, on Google's QAT base. |
The third row catches people out. A repository named qat-4bit is not a uniform 4-bit quant at all. It puts 144 components at 8 bits and the file lands 3.9 GB heavier than the flat quant above it. Nothing wrong with the choice. The name simply does not describe it, and scores compared across these three are scores at three different sizes.
The convention comes from llama.cpp, where Q4_K_M and its relatives name the predominant precision and not the average. Publishing the size alongside the score keeps that usable. A leaderboard column listing three "4-bit" quants does not.
Two roads to the same place, and they stack.
QAT is not something a user turns on. Somebody has to run quantization-aware training while the model is being trained, then publish those weights. Google does it for Gemma-4. Very few others do. If your checkpoint has no QAT weights, the first table is the comparison that applies to you.
Where QAT weights exist they work well. A uniform 4-bit quant of Google's QAT Gemma-4-12B scores 68.27. OptiQ on the plain instruct base scores 68.23. Those came from separate runs against different baselines, so treat them as level. Two techniques with nothing in common, landing on the same number.
They also stack. Run the sweep on the QAT base and it reaches 69.64, which is +1.37 over a uniform 4-bit quant of that same base.
| Build | Base | Capability Score |
|---|---|---|
| Uniform 4-bit | instruct | 61.83 |
| OptiQ mixed 4/8 | instruct | 68.23 |
| Uniform 4-bit | QAT | 68.27 |
| OptiQ mixed 4/8 | QAT | 69.64 |
The margin does shrink. OptiQ is worth +6.40 on the instruct base and +1.37 on the QAT base, because QAT has already taken out most of what the sweep goes looking for. The smaller Gemma-4 sizes track it: +2.09 on E2B, +1.19 on E4B.
If a model ships QAT weights, start with those. The sweep still adds about a point on top, and it costs an afternoon instead of a training run.
How these numbers were produced.
Six benchmarks, fixed sample counts, greedy decoding throughout so two quants of the same model differ only by their weights. MMLU at 1000 samples, GSM8K at 1000, IFEval on the full 540, BFCL-V3 simple at 200, HumanEval on all 164 with the programs executed in a sandbox, and HashHop at roughly 12k context.
Each quant is scored against a uniform 4-bit quant of its own base. That isolates the per-layer allocation from whatever else the base brings. It also means the two middle rows of the table above come from separate runs against different baselines. Read them as level.
MMLU reads low for this family. The default scoring takes the answer letter by logit argmax, which suits a model that answers directly and undersells one trained to reason first. Between two quants of the same base the number still discriminates, and that is all it does here. Do not line it up against an MMLU figure from somewhere else.
$ optiq eval mlx-community/gemma-4-12B-it-OptiQ-4bit \
--task all --score
Every quant carries its own table on its Hugging Face page, scored the same way against the same kind of baseline. The model list has sizes and RAM for all of them, and the eval-framework write-up covers why each benchmark is in the six and what the Capability Score does and does not tell you.