inabot

Measurements · Intel Arc Pro B60

Intel Arc Pro B60 for LLM inference: what four cards actually do

Three MoE models held in VRAM at once on 4× Arc Pro B60 24 GB, served by llama.cpp (SYCL) and vLLM-XPU. Measured on 10 and 11 September 2026 on a machine that also carries a production workload, so these numbers are a floor, not a ceiling. Every figure, script and failure of that series is in our public repository. On 11 November 2026 we added to this page a lab measurement of a single 27B model spread across the four cards.

93 tok/sQwen3.8-27B INT4 on 4 cards, single request, speculative decoding — 11 Nov 2026
685 tok/saggregate at 64 concurrent requests (12 per request), same model, no speculative decoding — 11 Nov 2026
211 Wpeak for the four cards, 163 W idle, 52–62 °C (September series, three models)
≈ CHF 25 / GBof VRAM at Swiss retail price (CHF 614 per 24 GB card)

Open the repository on GitHub →

Update · 11 November 2026

Qwen3.8-27B spread across the four cards

A lab measurement on the same machine: one dense 27B model in INT4, tensor parallel across the four B60, served by vLLM 0.32 built for XPU. It is not the configuration shipped with Anchor, which remains the three resident models described further down.

Concurrent requestsWith speculative decoding: aggregate / per requestWithout: aggregate / per request
193 / 93 tok/s59 / 59 tok/s
8316–330 / 45314 / 43
16404 / 30396 / 30
32480 / 17563 / 20
64488 / 8.5685 / 12
  • Setup: Qwen3.8-27B, INT4 weights, tensor parallel 4, FP8 KV cache, 131,072-token context, 300 generated tokens per request. Same host as below (i9-9980XE, 64 GB of RAM).
  • Cache reported by the engine with speculative decoding: 1,217,296 tokens, i.e. nine full 131k contexts held at once.
  • Speculative decoding (2 tokens ahead) wins up to about 16 concurrent requests and loses beyond. Highest aggregate measured: 685 tok/s at 64 requests without it, 12 tok/s per request; not tested beyond.
  • Limits: the first read of a long prompt runs at about 900 tok/s (110,000 tokens in about two and a half minutes). With speculative decoding, the first token on a 7,600-token prompt already in cache goes from about 1.3 s to 3.0 s.
  • Quality of the INT4 weights was only checked on an internal 22-question grid (21 correct, same as FP8): not a published benchmark.

As of 11 November 2026 this series is not in the public repository. An aggregate figure is the throughput of the whole machine, never of one user.

The machine

GPUs4× ASRock Arc Pro B60 Creator 24 GB (BMG-G21), each at PCIe x8 Gen3
HostIntel i9-9980XE, X299 (2018), 64 GB DDR4-3200
OS / driverUbuntu 24.04, kernel 7.0 HWE, xe driver, GuC 70.72.1
Runtimesintel/vllm:0.21.0-xpu and llama.cpp SYCL (server-intel image); vLLM 0.32 built for XPU for the November series

September 2026: three models resident at the same time

CardModelFormat / runtimeWeights
0gpt-oss-20bMXFP4 · vLLM-XPU13 GB
1Qwen3-Coder-30B-A3BGGUF Q4_K_XL · llama.cpp SYCL17.7 GB
2 + 3Qwen3-Next-80B-A3BGGUF Q3_K_XL · llama.cpp SYCL35.6 GB

130 B parameters combined, about 3 B active per token in each model. 81.7 GB of 96 GB in use, weights plus KV cache.

Single-model throughput, warm

ModelDecodePrefill (~4k tokens)
Qwen3-Coder-30B-A3B82.4 tok/s4 098 tokens in 4.2 s
gpt-oss-20b41.2 tok/s3 307 tokens in 2.4 s
Qwen3-Next-80B-A3B36.2 tok/s4 098 tokens in 10.3 s

The first call after loading is 3 to 4 times slower: send a warm-up request before real traffic.

Concurrency: where it scales and where it breaks

With the three models serving 12 simultaneous requests, 3 600 tokens came out in 30.3 s (119 tok/s aggregate), with no error and no model slowing another. Per model, the runtime decides everything.

gpt-oss-20b on vLLM-XPU (continuous batching)

Concurrent requestsPer requestAggregate
135.1 tok/s35.1
235.0 tok/s70.0
433.6 tok/s133.9
8—241

Qwen3-Next-80B on llama.cpp SYCL (no continuous batching)

Concurrent requestsPer requestAggregate
132.0 tok/s32.0
214.5 tok/s28.8
47.0 tok/s20.7

Correction, published in the open: our first concurrency run divided token counts by global wall-clock time. It flattered the fast models and hid the 80B regression. The tables above are the corrected figures.

Design rule we took from it: anything latency-sensitive (voice, interactive completion) goes to the model served by vLLM. A large model on llama.cpp serves about two concurrent users, not four.

Host resources and power

Idle, 3 models loadedUnder 12 concurrent requests
Host RAM available43.1 GB of 62 GB41.5 GB (trough)
VRAM used81.7 GB of 96 GBsame
Power, 4 cards163 W211 W peak
Temperatures52–62 °C52–62 °C

64 GB of host RAM is enough for this configuration. The constraint is at load time, not while serving: models must be loaded one after the other.

Eleven failure modes we hit

The part that is not documented anywhere else. Exact error strings and fixes are in the repository.

  1. vLLM-XPU only serves quantisations that have a native XPU kernel: MXFP4 works, most quantised MoEs fail on an NVIDIA-only kernel path.
  2. AWQ requires --dtype float16 explicitly.
  3. oneCCL could not exchange IPC handles between workers, which blocked tensor parallel with vLLM 0.21. Lifted in November 2026 with vLLM 0.32: see the update above.
  4. Two processes on the same GPU can wedge the host.
  5. Models must be loaded sequentially: parallel loads froze the machine three times.
  6. The first inference after load is 3 to 4 times slower.
  7. Docker --ipc host silently cancels shm_size.
  8. A card with its 8-pin connector unplugged is simply absent from the PCI bus.
  9. Battlemage cards show x1 in lspci: that is cosmetic, read the root port instead.
  10. GuC firmware 70.44.1 was unstable; 70.72.1 from kernel.org fixed it, without a reboot.
  11. Vulkan is not an alternative to SYCL for decode.

Which motherboards take four cards

Arc Pro blower cards are two slots thick, which rules out most server boards. Measured from vendor drawings: ASUS Pro WS WRX90E-SAGE SE and W790E-SAGE SE take four on 8 bracket positions; Supermicro H13SSL-N and H12SSL-i take only three. Full table in the repository.

Not measured yet

  • Power draw and temperatures of the November configuration
  • Arc Pro B70 and Arc Pro B60 Dual: no cards yet
  • More than 64 concurrent requests

Use the data

Code under MIT, documentation and measurement data under CC BY 4.0: use the numbers, cite the repository. If you find a mistake, open an issue.

Open the repository on GitHub

The same stack, delivered

SOKKAN Anchor is this machine, assembled in Geneva with the three models already running and an OpenAI-compatible API. You can test Qwen3.8-27B, the model measured in the November update, through our API with a trial key before buying.