Local AI Workstation · Build Sheet
Kirbotron
Single-socket AMD EPYC Milan workstation: 64 cores, 256 GB ECC memory, and 144 GB of GPU VRAM in a chassis you can put on a desk.
64 / 128
cores / threads · EPYC 7713
205 GB/s
DDR4-3200 · 8-channel theoretical peak
144 GB
total GPU VRAM (3 cards)
144 lanes
PCIe 4.0 from socket + chipset
01Parts
All figures from vendor spec sheets. Bandwidth numbers are theoretical peaks.
| Component | Spec | Notes |
|---|---|---|
| CPUAMD EPYC 7713 (Milan, SP3) | 64C/128T · 2.0–3.675 GHz · 256 MB L3 · 225 W TDP | Full 128 PCIe 4.0 lanes and 8-channel memory controllers on-die. |
| MotherboardSupermicro H12SSL-NT | 5× PCIe 4.0 x16 · 2× x8 · 2× M.2 · 2× SlimSAS ×8 | Dual 10GBase-T (BCM57416) on-board unused · AST2500 BMC / IPMI ready · ATX, up to 2 TB ECC |
| MemoryKingston Server Premier KSM32RD8 · 256 GB total | DDR4-3200 ECC Registered · 2Rx8 | 8× 32 GB · all 8 channels populated → ~204.8 GB/s peak (8 × 25.6 GB/s). |
| GPU ×2Zotac RTX 3090 (NVLink bridged) | 24 GB GDDR6X ea · 936 GB/s mem BW · NVLink 3 112.5 GB/s | 48 GB pool Paired for tensor parallelism over NVLink. |
| GPU ×1PNY RTX PRO 6000 Blackwell | 96 GB GDDR7 ECC · ~1.8 TB/s mem BW · 300 W TDP | 96 GB Single-card home for 176B-class models at NVFP4. |
| NVMeSamsung 990 PRO · 4 TB | PCIe 4.0 ×4 NVMe · M.2 | Model storage — weights land fast, checkpoints write fast. |
| HDDSeagate IronWolf Pro 3.5" · 12 TB | SATA3 NAS drive · via SlimSAS port | Media storage / serving. |
| NetworkTP-Link TX201 | 2.5 GbE · RTL8125 | Primary LAN. Adds one port over the idle 10 GbE pair. |
| ChassisASUS ProArt PA602 | E-ATX · 200 mm front fans | Workstation case sized for SP3 boards and full-length cards. |
| Power supplyASUS Pro WS 2200W | 2200 W · 80 PLUS Platinum · ATX 3.1 · 240 V input | Platinum efficiency is certified at 230 V — the reason this box runs on 240, not 120. |
| UPSEaton 9PX3000GLRT · outside the chassis | 3000 VA / 3000 W · online double-conversion · L6-20P | Double-conversion = the box never sees raw mains. Online UPS isolates the build from every mains anomaly. |
02PCIe topology
Everything downstream of one socket. No PCH bandwidth funnel — the GPUs never share a chipset link.
EPYC 7713
128× PCIe 4.0 + 8× DDR4-3200 controllers, on-die
▼ ▼ ▼ ▼
RTX 3090 #1PCIe 4.0 ×16 · slot
RTX 3090 #2PCIe 4.0 ×16 · slot
↔ NVLink bridge ↔ #1
↔ NVLink bridge ↔ #1
RTX PRO 6000PCIe 4.0 ×16 · riser
2× SFF-8654 cables
2× SFF-8654 cables
TX201 2.5GbESlimSAS #2 (SFF-8654)
On-board I/O2× 10GBase-T (idle) · 2× M.2 ×4 · SlimSAS #1 → IronWolf SATA · SATA3 · IPMI/BMC
Why the riser trick exists
Plain clearance math: the board offers seven PCI positions, and each Zotac 3090 is a 3-slot card. 3 + 3 leaves exactly one slot for the PRO 6000 — and a full-height x16 card can't sit there directly beside a 3090's cooler shroud. The riser takes the last open slot and hangs the PRO 6000 off two SFF-8654 cables instead. Most of the remaining slots are simply physically blocked by the 3090s. Both native SlimSAS ports are also in service: one feeds the TX201 NIC, the other carries the SATA link to the IronWolf.
03What it's good at
- Large-model LLM inference — the PRO 6000's 96 GB runs a 176B-class mixture-of-experts model at NVFP4 on one card with no offload.
- Tensor-parallel pairs — the NVLinked 3090s give 48 GB + 112.5 GB/s card-to-card for 2-way splits.
- CPU-side work — 64 cores + 205 GB/s memory handles data prep, tokenization, retrieval indexes, and quantization jobs while the GPUs cook.
- KVM-over-IPMI — headless management is built into the board; no monitor needed after setup.
04Watch-outs
Power budget with headroom
Rated TDPs sum to ≈1225 W (225 CPU + 2×350 GPU + 300 GPU + board/fans) against a 2200 W Platinum supply — comfortable, with room for transient spikes across three cards. The 240 V feed keeps the PSU in its 230 V efficiency band and halves the line current. Downstream, the 3000 W online-UPS gives roughly 2× the wall load for ride-through.
Memory bandwidth mismatch
GPU memory bandwidth (936 GB/s–1.8 TB/s per card) towers over system memory (205 GB/s). CPU offload of layers works but is the bottleneck — size models to fit VRAM, not RAM.
BIOS checklist
Above 4G Decoding on · PCIe bifurcation matched to the riser wiring · NPS (NUMA) setting chosen deliberately (NPS4 usually helps 8-channel locality) · C-states tuned for latency vs. idle power.
05Software stack
Everything that runs on the box, from the OS up to the traces.
| Layer | What runs | Notes |
|---|---|---|
| OSUbuntu LTS | Containerized workloads on a persistent data volume. | Everything that matters lives on the data volume; container state is disposable and rebuilt from source. |
| AgentHermes Agent | The control plane: tool calls, sessions, scheduled jobs, chat-platform gateways. | Orchestrates the GPUs — subagents fan out tasks across both inference engines. |
| InferenceSGLang + vLLM | OpenAI-compatible endpoints. SGLang serves a 176B-class Qwen mixture-of-experts model at NVFP4 on the PRO 6000; vLLM runs a Gemma-class model tensor-parallel across the NVLinked 3090s. | Two serving engines because each wins on different models and workloads. |
| MemoryHindsight | Long-term memory layer: fact extraction, entity resolution, semantic retrieval. | What lets the agent remember across sessions instead of starting cold every time. |
| ObservabilityPrometheus · Grafana · Tempo · Splunk | Prometheus scrapes GPU and serving metrics into Grafana dashboards; OpenTelemetry traces flow to Tempo; events and logs ship to Splunk. | Every agent turn and inference request is a trace, not a vibe. |
| Model library4 TB NVMe | Multi-model rotation without republishing weights. | Weights land fast, checkpoints write fast. |