$ pip install mlx-optiqone install · three tools
CLIoptiq convertquantize & serve
LABoptiq labthe workbench
CODEoptiq codecoding agent
Local LLMs on Apple Silicon
Run any LLM,
Run any LLM,
right on your Mac.
OptiQ is the local-LLM stack for Apple Silicon. Quantize open models down to a size your Mac can run, then use them three ways: from the command line, a local web app, or a coding agent in your terminal. No GPU cluster, no API key, no cloud.
100+
models on Hugging Face250k+
downloads / month · HFThe OptiQ stack
One local model. Three ways to put it to work.
OptiQ is a family of tools for running large language models on your own Mac. Quantize and serve them from the command line, drive the whole workflow from a local web app, or hand your model a coding task in the terminal.
CLI
OptiQ CLI
Quantize · fine-tune · serve
The optimizing compiler and toolkit. Sensitivity-driven mixed-precision quantization, sensitivity-aware LoRA, and a server that speaks both the OpenAI and Anthropic APIs, MLX-native, no PyTorch.
Read the CLI docs → LABOptiQ Lab
A local model workbench
One pip install for a browser UI over the whole workflow: chat with sandboxed tools and images, a quantize wizard, SFT and DPO fine-tuning with a dataset designer, and a two-model arena.
OptiQ Code
A coding agent for your model
A terminal coding agent that drives whatever you are serving, offline, on your Mac. Engineered for weak local models: never an empty patch, edit-resilient, and stall-proof.
Explore OptiQ Code →01 Pre-built models
Drop-in 4-bit quants. Same weights, smarter bits.
Production OptiQ-quantized LLMs on Hugging Face. Nemotron 3, MiniCPM5, Qwen3.5, Qwen3.6 and Gemma-4 families, from 1 B dense to 35 B-A3B mixture-of-experts. They load directly into stock mlx-lm. No special runtime.
Gemma-4 · new
gemma-4-12B-it-OptiQ-4bit
Google's unified text+vision Gemma-4, at 8.3 GB, with image input. Capability Score 68.2 (+6.4 vs uniform-4-bit), one of our largest mixed-precision gains, and the strongest model we ship under 9 GB on disk.
8.3 GB on disk
68.2 Capability
+6.4 vs U4
Gemma-4
gemma-4-31B-it-OptiQ-4bit
The largest single quant we ship. 31 B parameters in 20.8 GB with Capability Score 79.7 (+3.5 vs uniform-4-bit). Pair with the matching -assistant-bf16 drafter for speculative decoding.
20.8 GB on disk
79.7 Capability
+3.5 vs U4
Qwen3.6
Qwen3.6-27B-OptiQ-4bit
Frontier-class reasoning at 17.5 GB with our highest Capability Score (83.0). Bundled MTP head gives ~1.4× decode via optiq serve --mtp.
17.5 GB on disk
83.0 Capability
+0.5 vs U4
Qwen3.5
Qwen3.5-9B-OptiQ-4bit
The default daily-driver. 9 B parameters in 6.6 GB. Capability Score 66.8 (+0.2 vs uniform-4-bit). Long context to 64 k via mixed-precision KV; bundled MTP head for speculative decoding.
6.6 GB on disk
66.8 Capability
+0.2 vs U4
05 How it compares
Where mlx-optiq sits among the Mac LLM options.
A snapshot of how the popular paths stack up on the things that actually move quality and speed on Apple Silicon. None of these are wrong; they're optimizing different axes.
| mlx-optiq | mlx-lm | llama.cpp | |
|---|---|---|---|
| Per-layer mixed-precision weights | Yes, calibration-driven | Uniform 4-bit | Block-wise K-quant |
| Per-layer mixed-precision KV cache | Yes | Uniform 4 / 8 / fp16 | Group-wise int8 only |
| Sensitivity-aware LoRA fine-tuning | Rank scaled by per-layer bits | Constant rank LoRA | Inference only |
| OpenAI and Anthropic compatible server | One process, both | OpenAI only | llama-server (OpenAI shim) |
| Text and image input | Yes | Text only | Image via separate build |
| Sandboxed tool support for chat | Three tools: web search, Python, terminal | — | — |
Reading the table
mlx-optiq is the only path on this list that uses calibration-driven, per-layer bit allocation for both weights and KV cache, in the native MLX runtime, with serving, fine-tuning, and image input on the vision models in the same package. The others are great at what they target. They just target different things.
06 FAQ
Running LLMs on a Mac, answered.
Can I run LLMs locally on a Mac?
Yes. mlx-optiq runs large language models natively on Apple Silicon, from M1 to M5, using Apple's MLX framework. Install it from PyPI with pip install mlx-optiq, then quantize, fine-tune, and serve models entirely on your Mac, fully offline.
Do I need a GPU or PyTorch to run LLMs on a Mac?
No. There is no PyTorch and no discrete GPU in the path. mlx-optiq is MLX-native and uses the unified memory of Apple Silicon directly, so a MacBook, Mac mini, or Mac Studio is enough. No CUDA, no cloud, no API key.
How much RAM do I need to run an LLM on a Mac?
It depends on the model. A 4-bit OptiQ quant of a 4B model needs roughly 3 GB; a 9B needs about 6 GB; larger mixture-of-experts models need more. Mixed-precision 4-bit quantization is what lets bigger models fit in a Mac's memory while staying close to full-precision quality.
What is mlx-optiq?
An MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon. Its core is data-driven mixed-precision quantization: it measures each layer's sensitivity and assigns per-layer bit-widths, so quants keep more quality than uniform 4-bit at the same size. It also ships a local web UI (OptiQ Lab) and an OpenAI and Anthropic compatible server.
Get started
Make your Mac an LLM workstation.
Pick a model, get a snippet, ship it. The docs cover every supported family, fine-tuning recipes, and the OpenAI-compatible serving stack.