mlx-optiq
Documentation

Documentation

These docs cover OptiQ's three products and the engine under them. Code examples are copy-paste runnable on a Mac with Python 3.11+ and 16 GB+ RAM.

The products

  • OptiQ Cloud: open models for coding agents, paid per token: GLM 5.3 Flash, DeepSeek V4.1 Flash and GLM 5.3. Use them from Claude Code, Codex or another agent, from the API, or with Boost in OptiQ Code.
  • OptiQ Code: a terminal coding agent that runs on a local model or a cloud one.
  • OptiQ Lab: a local web app for chat, quantizing, fine-tuning and building datasets in the browser.

All three run on mlx-optiq, one pip install mlx-optiq. Its CLI quantizes, prunes, fine-tunes and serves models on Apple Silicon, using a per-layer sensitivity measurement to pick bits for each layer.

Pick a path

I want to run my coding agent on a cloud model

Sign in once with optiq cloud login, then one command runs Claude Code, Codex, OpenCode, Mistral Vibe, Hermes Agent or OpenClaw on a cloud model. New accounts get $2 of credit. See Use with coding agents.

I want to use a pre-built quant

Start with Installation, then jump to your model family: Nemotron, Qwen3.5, Qwen3.6, MiniCPM5, Spark-X2.5, or Gemma-4. Each has a 5-minute hello-world plus model-specific tips (chat template, sampling defaults, recommended context length).

I want to code with OptiQ Code

Serve a model, or pick a cloud one, then point OptiQ Code at your repo. It drives the served model through a read / edit / run-tests loop in your terminal, engineered to stay reliable on small local models. See the product overview.

I want to quantize my own model

Read How sensitivity works to understand the algorithm, then the convert CLI reference. The --reference auto flag picks bf16 when it fits and a uniform-4-bit baseline when it doesn't.

I want to fine-tune with LoRA

The LoRA fine-tuning guide covers PEFT-compatible adapter output, sensitivity-aware rank scaling, and the empirical training-ceiling map for a 36 GB Mac across all 12 supported models.

I want to serve an LLM

The Serving guide covers running optiq serve with both the OpenAI /v1/chat/completions and Anthropic /v1/messages endpoints from the same process, plus mixed-precision KV cache and a mounted LoRA adapter.

I want to send images

The vision guide covers image+text on Gemma-4, Qwen3.5 and Qwen3.6: one bf16 sidecar makes the same repo load text-only under stock mlx-lm and full image+text under OptiQ, with no mlx-vlm dependency. Upload a picture in the Lab, or send an image_url to optiq serve.

I want to evaluate a quant

The eval CLI ships a smoketest (KL + GSM8K-50, ~5 min on 27B) for triage and a full benchmark suite (MMLU + GSM8K + IFEval + BFCL + HumanEval + HashHop, ~2 h on 27B) that produces the Capability Score on every model card. HumanEval runs in a layered sandbox (container via Docker, Podman or Apple container → macOS sandbox-exec → subprocess + rlimit). HashHop checks long-context multi-hop retrieval via chained key/value lookups. Methodology is in the eval-framework write-up.

For agents and IDEs

The full library reference is also published as a single Markdown file: /llms.txt. Drop it into Claude Code, Cursor, or any agent context window.