Four 4-bit quants of Gemma-4-12B
Three repositories on Hugging Face put 4bit in the name of the same Gemma-4-12B. They run from 6.3 GB to 10.2 GB and no two quantize it the same way. Here is what each one scores.
What the extra 2 GB buys
Both start from google/gemma-4-12B-it. One gives every layer 4 bits. The other measures each layer first and raises the ones that cannot take it.
| Benchmark | OptiQ mixed 4/8 | Uniform 4-bit | Delta |
|---|---|---|---|
| MMLU | 42.6% | 34.4% | +8.3 |
| GSM8K | 93.4% | 90.1% | +3.3 |
| IFEval | 73.9% | 71.2% | +2.8 |
| BFCL-V3 simple | 71.0% | 71.5% | −0.5 |
| HumanEval | 88.4% | 76.8% | +11.6 |
| HashHop | 40.0% | 27.0% | +13.0 |
| Capability Score | 68.23 | 61.83 | +6.40 |
| On-disk size | 8.3 GB | 6.3 GB | +2.0 |
Six points on the mean, for 2.0 GB more on disk. BFCL goes the other way by half a point. At 200 calls that is inside the confidence interval.
How QAT compares
QAT is not a switch you flip. Somebody has to run quantization-aware training while the model trains, then publish those weights. Google does it for Gemma-4. Very few others do.
| Build | Base | Capability Score |
|---|---|---|
| Uniform 4-bit | instruct | 61.83 |
| OptiQ mixed 4/8 | instruct | 68.23 |
| Uniform 4-bit | QAT | 68.27 |
| OptiQ mixed 4/8 | QAT | 69.64 |
Those two middle rows came from separate runs against different baselines, so read them as level. They stack: the sweep on the QAT base reaches 69.64. The margin is smaller there, +1.37 against +6.40. QAT has already taken out most of what the sweep goes looking for.
gemma-4-12B-it-qat-4bit is not a uniform 4-bit quant either. It pins 144 components at 8 bits and weighs 10.2 GB. Comparing scores across the three published repos means comparing three different sizes.
Six benchmarks at fixed sample counts, greedy decoding throughout. Each quant is scored against a uniform 4-bit quant of its own base. That isolates the bit allocation from everything else the base brings.
MMLU reads low for this family because its default scoring takes the answer letter by logit argmax. It still separates two quants of the same base. That is all it does here.
$ optiq eval mlx-community/gemma-4-12B-it-OptiQ-4bit --task all --score
Every quant carries the same table on its Hugging Face page, all of them linked from the model list. The eval-framework write-up covers the limits of the Capability Score.