Why not just use Vulkan? It works out of the box.

Vulkan needs no patch, but at 8192 context it does 354 t/s prefill vs ROCm 653 t/s. That is 23.1s vs 12.5s prefill, net 35.6s vs 27.3s full cycle. Below ~1760 tokens Vulkan is net-faster, above that ROCm wins. For long-context research and agentic work, ROCm is clearly better once running.

Is forcing max_blocks_per_sm = 1 safe?

Unknown. That is why it is reported as bug #26702, not a fix. It avoids the crash and benches stably across 3 runs, but only a maintainer can confirm correctness and occupancy for gfx1031. Do not treat the one-liner as an upstream fix.

Does this explain the decode being slower on ROCm?

No. llama-bench excludes sampling time, so the TOP_K unsupported on ROCm log seen in llama-server is not confirmed as the cause of the tg512 gap here. That TOP_K CPU fallback was a side effect of the gfx override. It is indirectly fixed in EXP 009 by running natively without override.

Found the Bug, Fixed the Stack: ROCm gfx1031 vs Vulkan on RX 6700 XT.

Home Profile Experiments Contact Experiment 001 Experiment 002 Experiment 003 Experiment 004 Experiment 005 Experiment 006 Experiment 007 Experiment 008 Experiment 009 Experiment 010 Experiment 011 Experiment 012 Experiment 013 Experiment 014 Experiment 015 Experiment 016 Experiment 017