A world model you can drive, hidden in mlx-optiq

Run optiq game. It pulls 86 MB from the Hub and opens a window with a kart on a dirt track. You steer with the arrow keys. The track bends, trees slide past, the lap timer counts up in the corner.

None of that exists. There is no track geometry, no physics step, no renderer. A 131M-parameter latent diffusion model looks at the last seven frames and the keys you are holding, predicts the next latent, and a VAE decodes it to pixels. Then it does the same thing again, about seven times a second, for as long as you keep driving.

The HUD is what makes people look twice. Lap timer, position counter, minimap, speedometer dial. The model paints every one of them, because it saw them all in training. It does not know what a lap is. It knows what that corner of the screen usually looks like next.

Why it holds together

Most video models drift. You drive for ten seconds and the world turns to soup. This one holds together far longer than it should. The reason is the training data. The architecture is ordinary.

It was trained on SuperTuxKart, driven headlessly through pystk, which means every action label is ground truth read straight out of the simulator instead of guessed back from the video afterwards. Nothing was inferred. When the model learned that holding left bends the world left, it learned it from a label that was correct on every single frame.

The sampler is a rectified flow distilled down to two steps. Two is not a quality dial. The checkpoint is a student trained at k=2, and this model family comes apart at k=4 and above. There is no key binding to raise it, on purpose.

The quantization is the point

The published weights are OptiQ mixed 4/8-bit. That is what gets the download to 86 MB. Size matters more here than it does for a chatbot. You have a frame budget. Seven frames a second leaves about 140 ms to predict a latent and decode it, and the VAE already eats a chunk of that. Shrinking the DiT buys the rest back.

It also means the whole thing fits in the same install as everything else. No separate download step, no CUDA, no server. The weights land in the Hugging Face cache the first time you run it and resolve from there afterwards, so the second launch is instant and works with the wifi off.

One small design note

The control legend along the bottom of the window is also the input indicator. Each chip lights up while its key is held.

That exists because of latency. At seven frames a second the world reacts a frame or two after you press something. Players read that as a dropped key. So every control stays visible and lights on press. The chips come from the action vector going into the model, never from the keyboard. A chip that sticks on is a real bug, not a display quirk.

Try it

$ pip install mlx-optiq
$ optiq game

Arrow keys to drive, R to respawn somewhere else on the track, Q to quit. It needs Apple Silicon, since it runs on MLX.

The quantized weights are at mlx-community/neural-drive-optiq-mlx. The original model is at codelion/neural-drive-model. There is a browser demo in this Space if you want a look before installing anything.

It is an easter egg. It is also the shortest way to show what the toolkit is for. Someone else trained the model. We squeezed it to 86 MB. It still drives.