The local LLM hardware race is heating up around a 4-GPU desktop rig, built to squeeze in-house inference with 7 PCIe 5.0 lanes and 4 PCIe 5.0 SSDs for model storage. The plan leans on a beefy motherboard, a Threadripper Pro CPU, and serious power to keep the GPUs fed. [1]
• ASUS Pro WS WRX90E-SAGE motherboard — 7 PCIe 5.0 lanes to support multiple GPUs and fast storage. [1] • Ryzen Threadripper Pro 7995WX CPU — ordered, a cornerstone for this setup. [1] • PCIe 5.0x4 SSDs — for model storage, with RAID 0 envisioned. [1] • SilverStone Alta D1 case — chosen to fit the monster build. [1] • ASRock TC-1650T 1650 W Power Supply — dual-PSU thinking to feed the GPUs and motherboard. [1]
Power and cooling aren’t afterthoughts here. The plan assumes around 600W per GPU, with dual PSUs and even the potential need for a 240V outlet. The debate also pits Max Q-style cooling against full workstation cooling for reliability. [1]
Beyond parts, the discussion weighs feasibility, cost, and ROI of local inference versus cloud. The build hints at a 30k+ price tag, and benchmarking is on the docket to prove whether on-site inference pencils out. [1]
Benchmarking and hands-on results will shape whether this kind of local rig becomes mainstream or stays in the enthusiast lane. [1]
References
More money than brains... building a workstation for local LLM.
Member plans a 4x RTX/6000-class build for local LLMs, debating CPUs, RAM, PCIe lanes, power, cooling, and performance benchmarks ahead.
View source