TRBuy
Loading 3D model

RACK-GPU PRO 128GB / AI INFRASTRUCTURE WORKSTATION

Rack-GPU Pro 128GB

On-prem LLM inference: 4 × Radeon AI PRO R9700, 128 GB of VRAM, 2,560 GB/s.

Quoted price$19,750+ VAT$23,700 incl. VAT

System architecture · 4U

One host, four PCIe Gen5 x16 accelerators.

On the TRX50 platform each R9700 has its own root complex and a full x16 link. The host side is a Threadripper 9960X with 64 GB DDR5 ECC; power comes from one 2,500 W ATX 3.1 PSU.

One root complex per card
4 × Gen5 x16
128 GB VRAM + 64 GB DDR5
192 GB
ATX 3.1 · 80 PLUS Platinum
2,500 W
186 × 445 × 455 mm · rack rails
4U

Accelerators

4 × Radeon AI PRO R9700

GDDR6 · 4 × 32 GB · 256-bit per card
128 GB
Aggregate bandwidth · 4 × 640
2,560 GB/s
Navi 48 · RDNA4 · gfx1201
256 CU
INT4 theoretical · 4 × 191
764 TFLOPS

Decode · memory-bound

Decode speed is capped by memory bandwidth.

Every decode step reads all active weights from VRAM; step time is set by read speed, not FLOPS. In a batch that one read is shared by every concurrent request, which is why throughput grows with concurrency.

tok/s≈bandwidth÷bytes / token

Tensor parallel · TP=4

Every layer is split across four cards.

Weight matrices are cut into four shards; each card computes its quarter, and the partial results are combined at the end of the layer with an RCCL all-reduce.

bf16 measured · 10 min
382.3 TFLOPS
All-reduce · 32 MiB
40.0 GB/s
Card-to-card spread
0.5%

Host · second memory tier

128 GB VRAM + 64 GB DDR5 = 192 GB.

Threadripper 9960X (24C/48T, 4.2–5.4 GHz) with DDR5-5600 ECC RDIMM. Tensors that do not fit in VRAM stay in host RAM and are read by the GPU directly over UVA: on the test system a 169 GB model was served from 112 GB of VRAM plus 70 GB of host RAM; that model needs a RAM upgrade.

VRAM · 4 × 32 GB GDDR6
128 GB
Host RAM · DDR5-5600 ECC
64 GB
Usable total
192 GB

Power · ATX 3.1

2,500 W, a native 12V-2x6 run per card.

One ATX 3.1 PSU, 80 PLUS Platinum, fully modular. Each R9700 draws from its own cable; no adapters, no splitters. ATX 3.1 rides through short power excursions of 2× in total and 3× on the GPU connectors.

Native · one cable per card
4 × 12V-2x6
Card TBP / connector rating
300 / 600 W
Efficiency, typical load · Platinum
≥ 94%
AC input · 15 A · single feed
200–240 V

Airflow · front-to-back

One airflow path: intake at the front, exhaust at the back.

3 × 140 mm fans pull through the honeycomb front panel. The R9700 are blower cards: they push their heat out through the slot bracket instead of into the chassis, so four cards run side by side at 2-slot pitch.

Front intake
3 × 140 mm
Blower, out the bracket
4 × 300 W
Peak junction · no throttling
105 °C

Components

Quoted configuration

Processor
AMD Ryzen Threadripper 9960X24C / 48T · 4.2–5.4 GHz · 350 W · sTR5
Memory
64 GB DDR5-5600 ECC RDIMM2 × 32 GB
Accelerators
4 × AMD Radeon AI PRO R970032 GB GDDR6 · 256-bit · 300 W · blower · 2-slot
Motherboard
TRX50 · sTR5 · E-ATXPCIe Gen5 x16 per GPU
Storage
1 TB NVMe PCIe Gen4M.2 2280
Power supply
2 500 W ATX 3.1 · 80 PLUS Platinum4 × native 12V-2x6 · fully modular
Chassis
Rack-GPU 4U3 × 140 mm · 8 PCIe slots · 186 × 445 × 455 mm

Operations

Configured for unattended operation.

  • systemd service · starts at boot · Restart=on-failure
  • wait-amdgpu.sh · service waits until all four cards are ready
  • D3 disabled · sensor readout and P2P continuity kept
  • CPU governor: performance, persistent
  • RCCL preflight · P2P/IPC · 40 GB/s
  • OpenAI-compatible · /v1/completions · /v1/chat/completions

Assembly and test

Assembled and tested by Radius.

The R9700 cards are in stock. About one week from order to delivery, assembly and testing included.

Rack-GPU Pro 128GB

  • 4 × R9700
  • 128 GB GDDR6
  • 64 GB DDR5 ECC
  • 2,500 W ATX 3.1
  • 4U

Engineered by Radius. Designed in Türkiye.

Quoted price$19,750+ VAT$23,700 incl. VAT

Buy on RackDepoKernel LayerSpecifications

Questions: 0 (216) 344 54 64

Software · Radius kernel layer

Inference kernels written for RDNA4.

vLLM's MoE, int4 and hybrid-attention paths were written for CUDA; on RDNA4 their ROCm counterparts were missing or slow. Radius rewrote those paths in HIP for gfx1201. In the measured run, 30% of total kernel time is spent in these kernels.

Inference stack
  1. OpenAI-compatible API/v1/chat/completions · /v1/completions
  2. vLLM 0.30continuous batching · prefix cache · TP=4 + EP
  3. Radius kernel layerRAVEN · SUPER · TALON · SHRIKE · ple_host
  4. ROCm 10 · RCCLPyTorch 2.13 · Triton 3.8
  5. Debian 13 · Linux 6.12amdgpu 7.1.3
  6. 4 × R9700Navi 48 · PCIe Gen5 x16

Opssystemdwait-amdgpu.shD3 offgovernor: performanceRCCL preflight

One decoder layer, per card
GPU · TP=4 shardhidden state · bf16
Token mixerGDN · QSA sparse attentionSUPERSHRIKE
Linear layersint4 W4A16 · narrow bf16 matricesRAVENTALON
MoE FFN512 experts · int4RAVEN
All-reduceRCCL · P2P · 40 GB/s
× 48 layers · 36 GDN + 12 QSA
Host RAM
UVA
PLE table47.7 GB · FP8ple_host

Qwen3.8-Flash-Next. The block for the kernel card being read is highlighted.

  1. RAVEN

    W4A16 MoE + linearHIP

    W4A16 GEMM: int4 weights, 16-bit activations. For MoE experts and the int4 linear layers.

    Weights are dequantised inside the kernel; the bytes read from VRAM per token drop to a quarter of bf16.

    Weights read per token

    bf16≈ 20 GB
    int4≈ 5 GB

    Qwen3.5-122B-A10B · ~10B active parameters / token

    Replaces
    vLLM's RDNA3, Triton and Exllama paths
    Measured
    8% + 7% of kernel time

  2. SUPER

    QSA sparse attentionHIP

    QSA sparse attention in HIP, registered with vLLM as a CUSTOM attention backend.

    The QSA layers of hybrid models run inside continuous batching; requests do not queue one by one.

    Replaces
    The standard ROCm attention path (missing or slow for hybrid attention)
    Measured
    8% of kernel time

  3. TALON

    bf16 skinny GEMMHIP

    bf16 skinny GEMM: small M, wide K.

    At TP=4 each card gets a quarter of every matrix, and in decode M is only as large as the batch. General-purpose GEMM tiles leave CUs idle on these shapes.

    Replaces
    General-purpose bf16 GEMM
    Measured
    7% of kernel time

  4. SHRIKE

    GDN prefill fusionHIP

    GDN prefill fusion: 3 Triton kernels → 1 HIP kernel.

    Intermediates are no longer written to global memory and read back, and two launches disappear; TTFT drops on long prompts.

    Replaces
    3 separate Triton kernels
    Measured
    1.24–1.94× speed-up, no regression

  5. ple_host

    FP8 PLE table · host RAM · UVAvLLM plugin

    vLLM plugin: the PLE table in FP8, in host RAM, read from the GPU over UVA.

    The 95.4 GB bf16 table (320,001,536 × 160) is written straight from the checkpoint shards to FP8. No bf16 copy is ever made, no D2H sync is needed, and HIP graph capture stays intact.

    169 GB model, memory at runtime

    • VRAM · 4 × 28 GB · weights + KV + graph
    • Host RAM · 4 workers × 17.5 GB · PLE FP8, UVA

    182 GB in total: more than a single 128 GB pool holds. · 95.4 GB bf16 → 47.7 GB FP8

    Replaces
    The standard path that loads the bf16 table as-is
    Measured
    VRAM increase 0.0 MiB · FP8 relative error 2.21%

Kernel profile

Share of total kernel time, measured run.

Measured performance

122B MoE, TP=4: 1,550 tok/s.

Qwen3.5-122B-A10B int4 · 1k in / 1k out · cc 1–128 · vllm-bench serve, 10 October 2026 · 8/8 cells.

Output throughput · cc=128
1,550tok/s
Single user · cc=1
84.2tok/s
TTFT · cc=1
192ms
Energy efficiency · cc=128
1.29tok/J
Output throughput by concurrency (tok/s)Rack-GPU Pro · Qwen3.5-122B-A10BASUS Ascent GX10 · gpt-oss-120B
04008001,2001,6001248163264128Concurrent requests (cc)1,550.2Rack-GPU Pro564.7Ascent GX10

The model choice works against Rack-GPU: 122B-A10B reads ~10B active parameters per token, gpt-oss-120B ~5B. At cc=128 throughput was still climbing (+27.5%); saturation was not reached.

What concurrency costs

As cc rises, total throughput grows while per-user speed falls and TTFT lengthens. GPU power holds at 1,197 W, the board-power limit, from cc=16 on.

Per user (tok/s)cc 1 → 12885.5 → 12.6
030609018128
Mean TTFT (ms)cc 1 → 128192 → 2,891
01,0002,0003,00018128
GPU power (W)cc 1 → 128756 → 1,198
04008001,20018128TBP 1,200 W
As a table
Concurrency sweep · 122B · 1k/1k
ccOutput tok/sTotal tok/sPer-user tok/sMean TTFT msGPU power W
184.2168.485.5192756
2111.4222.956.4249783
4260.4520.866.4315989
8403.0806.151.85611,111
16619.41,238.840.08251,197
32884.11,768.228.51,1621,197
641,215.82,431.719.61,7151,197
1281,550.23,100.412.62,8911,198

Long context: 169 GB Flash-Next

Qwen3.8-Flash-Next W4A16, 36,864-token window, ple_host. With long inputs most processed tokens are prefill; at 32k input, single-user decode runs at 55 tok/s.

Output (decode)Input (prefill)

  • 1k / 1k cc=161,021 tok/s totalTTFT 1,101 ms
  • 4k / 2k cc=161,404 tok/s totalTTFT 2,783 ms
  • 8k / 2k cc=162,021 tok/s totalTTFT 5,123 ms
  • 16k / 4k cc=1284 tok/s totalTTFT 1,993 ms
  • 32k / 4k cc=1496 tok/s totalTTFT 4,006 ms
As a table
Input / outputccOutput tok/sTotal tok/sMean TTFT ms
1k / 1k16510.31,020.71,101
4k / 2k16468.01,404.02,783
8k / 2k16404.22,021.15,123
16k / 4k156.8284.11,993
32k / 4k155.1496.04,006

Prefix cache A/B

4k/2k, cc=16. In the second run 50% of requests share a prefix (system prompt, RAG context).

  • TTFT−29%2,783 ms1,984 ms
  • Queue time−40%1,323 ms792 ms
  • Inter-token latency p99−87%243 ms31 ms
  • Output throughput+6.5%468.0 tok/s498.2 tok/s

No shared prefix50% shared prefix

Hardware validation: 10 min at full load

bf16 TFLOPS per card0.5% spread
  • GPU095.3
  • GPU195.6
  • GPU295.7
  • GPU395.8

GPU power, 122B full load1,198 W

TBP 4 × 300 W · 99.8%

Peak VRAM per card26.8 GB

32 GB · 83.8%

bf16 total, sustained
382.3 TFLOPS
All-reduce · 32 MiB · RCCL P2P
40.0 GB/s
Peak junction · no throttling
105 °C
CPU fp64
1,780 GFLOPS

Test system “sage”: ASUS Pro WS TRX50-SAGE WIFI, Threadripper 9970X (the quoted configuration has the 9960X and 64 GB of RAM), 4 × R9700, Debian 13 · ROCm 10.0 · vLLM 0.30. Every run is kept as JSON + CSV + logs with 1 Hz power, junction temperature and VRAM telemetry.

Comparison

Rack-GPU Pro and ASUS Ascent GX10

Ascent GX10 shares one 128 GB LPDDR5x pool between CPU and GPU at 273 GB/s. Rack-GPU Pro drives four cards' 640 GB/s GDDR6 in parallel and adds an independent 64 GB DDR5 tier (up to 512 GB). Decode is bandwidth-bound, so the gap shows up directly in token speed.

Memory bandwidth9.4×
Rack-GPU Pro 128GB2,560 GB/s
ASUS Ascent GX10273 GB/s
Peak output throughput, 120B class1.8×
Rack-GPU Pro 128GB1,550 tok/s
ASUS Ascent GX10863 tok/s
Single-user speed, 120B class2.5×
Rack-GPU Pro 128GB84.2 tok/s
ASUS Ascent GX1033.5 tok/s
Usable memory, VRAM + host RAM1.5×
Rack-GPU Pro 128GB192 GB
ASUS Ascent GX10128 GB

Paths to 128 GB of VRAM

Türkiye retail, VAT included, October 2026. Card prices cover the cards only; the Rack-GPU price is the complete system.

Paths to 128 GB of VRAM
Paths to 128 GB of VRAMPrice, VAT incl.Card powerDelivery
Rack-GPU Pro 128GB4 × R9700 · complete system · 128 GB$23,7004 × 300 W≈ 1 week
4 × NVIDIA RTX 5090cards only · GeForce licence · 128 GB$31,000–32,0004 × 575 W1–3 months
1 × NVIDIA RTX PRO 6000card only · 96 GB$20,450–23,400600 W2–6 months

Decision matrix

NeedRack-GPU ProAscent GX10
Serving 10+ concurrent usersSuitedLimited
Batch document / data processingSuitedLimited
170 GB+ modelsWith a RAM upgradeDoes not fit
Dense 49–70B modelsFull speed3–6 tok/s
Long context (32k+), RAGSuitedLimited
Customer-facing TTFTSuitedLimited
Growing the hardwareCards / RAM / disksSecond unit only
Architecture-specific kernelsOpen platformClosed product
Single-developer prototyping—Suited
Desktop / portable—1.6 L

Rack-GPU figures are Radius measurements. Ascent GX10 figures come from published independent sources (LMSYS, Dendro Logic, NVIDIA Developer Forums); conditions are not identical. The bandwidth gap is exact from the official specifications; the size of the performance gap may vary by ±30%. On request, Radius will run the same models with its own tool on an Ascent unit.

Model compatibility

Which models run?

On unified memory, 128 GB is a hard ceiling. On Rack-GPU, whatever does not fit in VRAM moves to the host RAM tier; the ceiling is 192 GB with the standard 64 GB of RAM, and RAM can be upgraded to 512 GB.

Weight size and memory ceilings

Weights only; KV cache and working memory excluded.

Fits in VRAMNeeds the host RAM tier

128 GB · Rack-GPU VRAM · Ascent unified
Qwen3.8-Flash-Next W4A16169 GB
200B+ MoE>128 GB
Qwen3.5-122B-A10B int471.5 GB
gpt-oss-120B MXFP466 GB
Llama 3.1 70B (dense)~40 GB
Nemotron Super 49B (dense)~26 GB
192 GB · Rack-GPU VRAM + 64 GB host RAM
ModelSizeRack-GPU ProAscent GX10
Qwen3.8-Flash-Next W4A16169 GBRunsRunning · 2,021 tok/s total (8k/2k, cc=16) · with a RAM upgradeUnusableDoes not fit: nowhere to offload in unified memory
Qwen3.5-122B-A10B int471.5 GBRunsRunning · 1,550 tok/s outputLimited~55 tok/s theoretical ceiling (10B active)
Llama 3.1 70B (dense)~40 GBRunsComfortableUnusable2.7 tok/s single request (FP8, LMSYS)
Nemotron Super 49B (dense)~26 GBRunsComfortableUnusable5.8 tok/s single request (NVFP4, Dendro Logic)
200B+ MoE>128 GBLimitedBorderline: with the host RAM tierUnusableDoes not fit
gpt-oss-120B MXFP466 GBRunsFitsRunsRuns · 863 tok/s peak

Datasheet

Specifications

Processor
AMD Ryzen Threadripper 9960X · 24 cores / 48 threads · 4.2–5.4 GHz · 350 W
Memory
64 GB DDR5-5600 ECC RDIMM server memory (2 × 32 GB) · upgradable to 512 GB
Accelerators
4 × AMD Radeon AI PRO R9700 · 32 GB GDDR6 · 256-bit · 300 W · blower · 2-slot
Total VRAM
128 GB · 192 GB usable in total (VRAM + host RAM)
Memory bandwidth
2,560 GB/s (4 × 640 GB/s, in parallel)
Compute
4 × 64 = 256 CU · measured bf16 382.3 TFLOPS · theoretical INT4 764 TFLOPS
Card-to-card
PCIe Gen5 x16, each card on its own root complex · RCCL all-reduce 40.0 GB/s
Motherboard
TRX50 · AMD sTR5 · E-ATX workstation/server class
Storage
1 × 1 TB NVMe PCIe Gen4 SSD · expandable
Network
10 GbE
Power supply
2,500 W ATX 3.1 · 80 PLUS Platinum (230 V EU), ≥ 94% efficiency at typical load · fully modular · 4 × native 12V-2x6 · single +12 V rail, 208.3 A · 200–240 V AC, 15 A · 135 mm FDB fan
Chassis
Rack-GPU 4U · 3 × 140 mm high-airflow fans · 8 PCIe slot design · rack rails
Dimensions
186 × 445 × 455 mm (H × W × D) · chassis 6.1 kg
Software stack
Tested: Debian 13 · ROCm 10.0 · vLLM 0.30 · Radius kernel layer (as an installation service)
API
OpenAI-compatible /v1/completions · /v1/chat/completions
Warranty
Hardcare Basic
Delivery
About one week after the order, assembly and testing included

Buy

Rack-GPU Pro 128GB

Complete system: hardware, assembly and testing. Order on RackDepo; configuration questions go to sales.

Quoted price$19,750+ VAT$23,700 incl. VATQuote of 16 October 2026 · payment in advance · shipping paid by Radius

Buy on RackDepoTalk to sales

Questions: 0 (216) 344 54 64 · satis@radius.com.tr

Assembled Rack-GPU 4U chassis
  • Assembly and testing included
  • Delivery in about one week · R9700 cards in Radius stock
  • Hardcare Basic warranty
  • Shipping paid by Radius
  • Debian 13 + ROCm 10 + vLLM stack, as an installation service

Only need the chassis? See the Rack-GPU 4U chassis page.