$ pip install mlx-optiqoptiq servequantize & serve
LABoptiq labthe workbench
CODEoptiq codecoding agent
BOOSToptiq cloud loginfrontier when needed
Local LLMs on Apple Silicon
Run any LLM,
Run any LLM,
right on your Mac.
OptiQ is the local-LLM stack for Apple Silicon. Quantize open models down to a size your Mac can run, then use them from the command line, a local web app, or a coding agent in your terminal.
200+
models on Hugging Face1M+
downloads / monthThe OptiQ stack
Everything talks to the same server.
Compile a model once with the CLI, then point whatever you like at it: the Lab in a browser, the coding agent in your terminal, or the editor you already use. Boost rides on that same server, which is why none of them needs a plugin.
CLI
OptiQ CLI
Quantize · fine-tune · serve
The optimizing compiler and toolkit. Sensitivity-driven mixed-precision quantization, sensitivity-aware LoRA, and a server that speaks both the OpenAI and Anthropic APIs, MLX-native, no PyTorch.
Explore the CLI → LABOptiQ Lab
A local model workbench
One pip install for a browser UI over the whole workflow: chat with sandboxed tools and images, a quantize wizard, SFT and DPO fine-tuning with a dataset designer, and a two-model arena.
OptiQ Code
A coding agent for your model
A terminal coding agent that drives whatever you are serving, offline, on your Mac. Built for weak local models: it never returns an empty patch and recovers from failed edits.
Explore OptiQ Code → CLOUDOptiQ Cloud
Boost a hard turn
A frontier model on tap for the turns your local one cannot finish. Boost lives in optiq serve, so Claude Code, Codex, OpenCode, Hermes and the Lab all get it with no plugin, and a whole tool loop costs one Boost.
OptiQ Cloud
One hard turn shouldn’t cost you the session.
Your local model handles the everyday work. When a turn starts going in circles, Boost hands that single episode to a frontier model and gives control straight back, in the same editor. You pay for the few turns that need it instead of a second subscription.
Eighteen turns on your own model. One Boost, and your code never left the machine except for that episode.
$20for 200 Boosts
- A whole tool loop1 Boost
- Free to start20 Boosts
- Works withany coding agent
No card to try it. Boost lives in the server, so nothing you already use needs a plugin.
Pre-built models
Drop-in 4-bit quants at the same size.
Production OptiQ-quantized LLMs on Hugging Face, across 12+ families: Qwen, Gemma-4, Muse-Glimmer, Nemotron, Mistral / Devstral, MiniCPM5, Laguna, Nanbeige, and a discrete-diffusion Gemma. They run from 1 B dense models to 35 B-A3B mixture-of-experts. Most load directly into stock mlx-lm; a few vendored architectures register with a one-line import optiq.
Qwen · new
Qwen3.8-27B-OptiQ-4bit
The newest Qwen and the first we ship with image input, with a bundled MTP head for speculative decoding.
18.9 GB on disk
87.98 Capability
vision + MTP
Muse-Glimmer
Muse-Glimmer-30B-OptiQ-4bit
A 30B image-text reasoning model that thinks in a separate channel before it answers. Capability Score 87.36.
22.1 GB on disk
87.36 Capability
100% HashHop
Gemma-4
gemma-4-31B-it-OptiQ-4bit
The largest single dense quant we ship. 31 B parameters in 20.8 GB with Capability Score 79.7 (+3.5 vs uniform-4-bit). Pair with the matching -assistant-bf16 drafter for speculative decoding.
20.8 GB on disk
79.7 Capability
+3.5 vs U4
Nemotron
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-OptiQ-4bit
A Mamba-attention hybrid with a 128-expert MoE, 6 active per token. 22.8 GB on disk, down from 65.8 GB, and the MTP speculative head is preserved in the sidecar for optiq serve --mtp.
22.8 GB on disk
73.94 Capability
MoE + MTP
How it compares
Where mlx-optiq sits among the Mac LLM options.
A snapshot of how the popular paths stack up on what actually moves memory footprint and decode speed on Apple Silicon.
| mlx-optiq | mlx-lm | llama.cpp | |
|---|---|---|---|
| Per-layer mixed-precision weights | Yes, calibration-driven | Uniform 4-bit | Block-wise K-quant |
| Per-layer mixed-precision KV cache | Yes | Uniform 4 / 8 / fp16 | Group-wise int8 only |
| Sensitivity-aware LoRA fine-tuning | Rank scaled by per-layer bits | Constant rank LoRA | Inference only |
| OpenAI and Anthropic compatible server | One process, both | OpenAI only | llama-server (OpenAI shim) |
| Text and image input | Yes | Text only | Image via separate build |
| Sandboxed tool support for chat | Three tools: web search, Python, terminal | None | None |
Reading the table
mlx-optiq is the only path on this list that uses calibration-driven, per-layer bit allocation for both weights and KV cache, in the native MLX runtime, with serving, fine-tuning, and image input on the vision models in the same package. The others target different goals.
FAQ
Common questions.
Can I run LLMs locally on a Mac?
Yes. mlx-optiq runs large language models natively on Apple Silicon, from M1 to M5, using Apple's MLX framework. Install it from PyPI with pip install mlx-optiq, then quantize, prune, fine-tune, and serve models entirely on your Mac, fully offline.
Do I need a GPU or PyTorch to run LLMs on a Mac?
No. There is no PyTorch and no discrete GPU in the path. mlx-optiq is MLX-native and uses the unified memory of Apple Silicon directly, so a MacBook, Mac mini, or Mac Studio is enough. No CUDA, no cloud, no API key.
How much RAM do I need to run an LLM on a Mac?
It depends on the model. A 4-bit OptiQ quant of a 4B model needs roughly 3 GB; a 9B needs about 6 GB; larger mixture-of-experts models need more. Mixed-precision 4-bit quantization is what lets bigger models fit in a Mac's memory while staying close to full-precision quality.
What is mlx-optiq?
An MLX-native toolkit to quantize, prune, fine-tune, and serve LLMs locally on Apple Silicon. Its core is data-driven mixed-precision quantization: it measures each layer's sensitivity and assigns per-layer bit-widths, so quants keep more quality than uniform 4-bit at the same size. It also ships a local web UI (OptiQ Lab) and an OpenAI and Anthropic compatible server.
Get started
Make your Mac an LLM workstation.
Pick a model and grab a snippet. The docs cover every supported family, fine-tuning recipes, and the OpenAI-compatible serving stack.