A DeepSeek-V4-Pro distillation runs on a used 8 GiB RX 580 and reads an image in 4.3 seconds. The kernel change we built to make it faster was 53% slower than the build we already ship. We reverted it, and that is the interesting part.
The model is DeepSeek-V4-Pro-Qwen3.5-9B, a decensored build of a 9B student distilled from DeepSeek-V4-Pro in what the card calls Max Effect mode. It is a qwen35 hybrid: 24 linear-attention blocks and 8 full-attention blocks across 33 layers.
At Q4_K_M the weights are 5.63 GB. That fits an 8 GiB card with room for a 16K context — 5.0 GB of weights, 537 MB of KV cache, 241 MB of compute buffers. Nothing is offloaded to the CPU during generation.
| Measured here | Value | Note |
|---|---|---|
| Prefill | 110–165 t/s | depends on prompt shape |
| Decode | 7–10 t/s | 5.24 GiB read per token is the floor |
| Context | 16 384 | 8 192 leaves the card short |
| Vision, uncapped | 16.9 s | our build |
Vision, --image-max-tokens 1024 | 4.3 s | our build |
| Vision, projector on CPU | 62–67 s | why we do not do that |
The image path runs with -fa auto, which is the default. It does not run with -fa off: that forces the dense path, allocates 4.58 GB of attention buffers, and segfaults on this card before a single token is produced.
That is worth writing down because the failure looks like a model problem and is not. The same weights read the same image in the same second once flash attention is left alone.
The time is in the token budget, not the kernel. Nothing about the image encoder was slow. Handing it 1024 tokens instead of the uncapped default cut 16.9 s to 4.3 s without changing the answer we were checking.
“Our build” is not upstream llama.cpp. The binary this page measures already carries our own Metal work for this card: the tiled q8_0 matvec, the width-64 fixes, and gated fusion. So the comparison below is our build against our build with one more patch — not a patched build against a clean one.
The 72-wide padding in the vision projector was suspicious: the card's wavefronts are 64 wide, and a 72-token block straddles them. So we built a 96-wide zero-pad — 96 being one and a half wavefronts — and measured it against the build we ship.
| Vision, one image | Seconds | |
|---|---|---|
| Our build | 16.9 | kept |
| 72 → 96 zero-pad | 25.9 | +53%, reverted |
The patch is gone from the tree. It is in this page because a failed optimisation that is not written down gets rebuilt six months later by someone with the same idea.
Two models were asked to produce the same Three.js island. The 9B was faster and worse.
| Model | Time | Outcome |
|---|---|---|
| DeepSeek-V4-Pro 9B | 5m 19s | page never renders |
| Ling 3.0 tiny | 10m 04s | island renders |
The 9B's page throws in OrbitControls(camera, renderer) — the second argument is accepted by nothing and the constructor never returns. The scene tree is incomplete and the camera is never pointed at anything, so even without the exception there would be an empty viewport.
Ling's page needed exactly one line: lookAt(12, 2, 12). Ling also paid for a retry, because a 4 096-token output cap truncated its first attempt — which is why the honest comparison is 10m 04s, not the faster number its first run would suggest.
Verdict. Ling stays the default for writing artefacts. The 9B earns its place on long-prompt, short-answer work — the shape it was distilled for — not on generating a scene graph from scratch.
Separate these from the table above: these are the publisher's numbers, from vLLM NVFP4 and MTP-Q8_0 builds, not from this card.
| Claim | Value | Build |
|---|---|---|
| GSM8K | 94.50% | 4-run mean, NVFP4, vLLM |
| MMLU-Pro average | 90.53% | Math/Physics/Chemistry, MTP-Q8_0 |
| Format compliance | 97.2% | answered with one capital letter |
We did not reproduce any of them. They are on the page so a reader can see which numbers are ours and which are borrowed.
./bin/llama -m models/DeepSeek-V4-Pro-Qwen3.5-9B-Q4_K_M.gguf \
-ngl 99 -c 16384 -fa auto \
--image-max-tokens 1024Drop -fa auto and it segfaults. Add -c 8192 and the card runs out of room before the context is full. Raise the image token budget and the 4.3 s becomes 16.9 s.
Measured on one used RX 580 with 8 GiB, on 2026-11-08, with the display attached — prefill and decode share the card with compositing, so the t/s figures move by roughly ±10% run to run. The vision timings are single images, repeated until three runs agreed within 0.4 s.