Experiments
Numbers from the lab. Everything here was measured on hardware I own, with the method written next to the chart so you can disagree with it.
Reading a 1K-token prompt
tokens/sec · Qwen3-Coder-30B-A3B at IQ3_XXS on a 4 GB laptop GPU, with the integrated GPU helping. Higher is better.
Measured 2026-10-13 on my own machine. The llama.cpp figure is from 2026-10-07.
Table view
| Model | tokens/sec |
|---|---|
| My engine | 135.4 |
| My engine, two days earlier | 120.7 |
| llama.cpp Vulkan (512 tokens) | 24 |
Writing the answer, short context
tokens/sec · Same model and machine, single stream. Higher is better.
Measured 2026-10-13. The first public build is from 2026-10-08.
Table view
| Model | tokens/sec |
|---|---|
| My engine, now | 21.5 |
| My engine, first public build | 9.5 |
| llama.cpp Vulkan | 3.5 |
| llama.cpp, experts on CPU | 2.4 |
Writing speed as the context grows
tokens/sec · The number that matters for real work: speed once the conversation is long, not on a one-line prompt.
Measured 2026-10-13, same model and machine throughout.
Table view
| Model | tokens/sec |
|---|---|
| at 4K tokens | 19.1 |
| at 16K tokens | 12.8 |
| at 32K tokens | 8.6 |
Five coding tasks, time until all five pass
seconds, lower is better · End to end: the model writes code, the code runs in a sandbox, and it repairs until the tests pass.
Same machine throughout; every run attempted the same five tasks.
Table view
| Model | seconds, lower is better |
|---|---|
| My engine | 165 |
| llama.cpp, tuned | 332.6 |
| llama.cpp, defaults | 311.4 |