Why a Home Lab in the Age of ChatGPT?
After discovering ChatGPT in late 2022, I became obsessed with running LLMs locally. Cloud APIs are convenient, but I wanted:
- Complete control over models and data
- Local inference for privacy-sensitive work
- Experimentation platform without API costs
- Learning environment for understanding how these systems actually work
Plus, my wife is a neat freak who’s always organizing the house. I channel that same energy into organizing computing infrastructure. Different obsessions, same impulse.
The Setup
I built a home lab from spare parts:
- 4-core AMD 2.8 GHz processor (modest but functional)
- 16GB RAM
- Collection of mismatched drives
- Old tower case resurrected
- Proxmox hypervisor for VM/container management
Interesting parts:
- Rescued a water-damaged SSD and memory stick (dry them thoroughly, they often work!)
- Used ZFS for storage management (learning experience)
- Everything held together with duct tape and determination
What I Run
LLM Experimentation:
- Llama models with 4-bit quantization
- Slow on CPU, but functional for learning
- Costs nothing per token
- Complete privacy
Statistical Simulations:
- R/Python environments for thesis work
- Bootstrap simulations that run for days
- Better than tying up my main machine
Standard Services:
- File server (Samba + encrypted rclone to Google Drive)
- VPN for remote access
- RStudio Server for writing/analysis anywhere
- Apache Guacamole for web-based VM access
The Reality Check
Performance: CPU-only LLM inference is slow. But it works, and I learn how these systems actually function.
Reliability: Good enough for experimentation, not for production services. I don’t run critical infrastructure on spare parts.
Power: The server is usually underutilized. But when I need to run week-long simulations or experiment with quantized models, it’s invaluable.
Why This Matters
After ChatGPT, everyone’s using cloud APIs. That’s fine. But running models locally teaches you things cloud APIs hide:
- How quantization works
- Memory vs. compute tradeoffs
- What “inference” actually means
- Why context windows have limits
Plus: no API costs, no data leaving my network, complete control over model selection.
The Lesson
You don’t need a $10k GPU server to experiment with LLMs. A pile of spare parts and Proxmox gets you surprisingly far.
The cloud is convenient. But there’s value in understanding the substrate—actual hardware running actual compute.
Also: if you’re doing long-running statistical simulations for research, a home lab beats monopolizing your desktop for days.
Built from e-waste. Runs LLMs. Good enough.
Discussion