The best AI PC for local LLMs

I run my local models on a pair of modded 48GB RTX 4090s in a tower that is neither small nor quiet, so the new crop of desk-sized AI machines has been on my mind for a while. NVIDIA's DGX Spark, the Ryzen AI Max mini-PCs and Apple's Mac Studio all promise the thing my bench isn't: proper models in a little box you can forget is there. They're not cheap, and they're not all good at the same things.

This guide covers what these machines are, whether you're better off with a plain graphics card, the one spec that decides it, and which I'd buy in each situation.

What an AI PC is

It's a whole computer built around running AI locally, rather than a graphics card you slot into a PC you already own. The design decision they nearly all share is unified memory - one big pool of fast memory shared between the processor and the AI bits, instead of a separate chunk of VRAM on a graphics card. That's how a little machine can offer 128GB for models when even a top-end GPU tops out at 32GB.

Small, quiet, sips power, and holds far bigger models than a consumer graphics card ever could. The catch is in how fast it holds them, which is where most of this guide will spend its time.

Whether you need one at all

For a lot of people the answer is no. If the models you want to run fit in 24GB - and an awful lot of very good ones do at 4-bit - then a used RTX 3090 will run them faster, for a third of the money. Graphics cards have far more memory bandwidth, and bandwidth is what makes tokens appear quickly.

An AI PC starts making sense when you want to run big models - the 70B-and-up sort that simply won't fit on a consumer GPU - or when the machine has to be small, silent and low-power on a desk or a shelf rather than a whirring tower. There's also the tooling angle: if you're already deep in Apple's world, the Mac path comes with software that fits. If none of that describes you, a prebuilt tower with a proper graphics card, or a GPU rig you build yourself, will be faster and cheaper.

The one number that matters (and the one right behind it)

Two numbers, really, and they trade against each other.

Memory size decides which models fit at all. More gigabytes, bigger models. A 128GB machine will hold a 70B model with room to spare, which is the main reason to want one. Memory bandwidth - measured in gigabytes per second - decides how fast those models run. Every word a model generates means reading its whole self out of memory, so more bandwidth usually means more words per second.

The tension: these AI PCs give you loads of memory but, mostly, modest bandwidth compared to a graphics card. They'll fit a huge model that a GPU can't touch, then run it at a gentle reading pace rather than a torrent. Whether that trade suits you depends on whether you need the big models or the speed. Our VRAM calculator will show you what a given amount of memory fits.

The three to know

A caveat before the list: I haven't had any of these three on my own bench. My models run on the 4090s, so what follows is worked from the specs and from other people's published numbers, and where the two disagree I've said so on each machine's page.

• The NVIDIA DGX Spark gets the most attention - 128GB, the full NVIDIA software world, aimed squarely at developers. Its bandwidth is modest, so think "runs enormous models" rather than "runs them fast".
• The Ryzen AI Max mini-PCs (the Framework Desktop and friends) cost the least per gigabyte. It's a normal x86 PC, so it runs everything, with up to 128GB of unified memory for a good deal less money.
• The Mac Studio has the best memory bandwidth of the lot, so it should be the fastest of these three on models that fit, and Apple's MLX tooling is well regarded - as long as you're happy living in macOS.

Which should you buy

If you live in the NVIDIA/CUDA world already and want the biggest models you can get, the DGX Spark. If you want the most memory for your money on a machine that runs anything, the Ryzen AI Max mini-PC - probably the best value here, and the one I keep circling when I imagine replacing the tower, though I can't justify it while the 4090s keep working. If you're an Apple household and want speed with big memory, the Mac Studio.

If you got this far and what you want is fast answers on models that fit in 24GB, you don't want an AI PC at all. You want a graphics card. Have a look at the GPU finder and save yourself a fortune.

Before you spend

• Don't buy on memory size alone. A big number with slow bandwidth runs big models slowly. Match the machine to whether you care more about model size or speed.
• Check the software fits your world. Some tools are NVIDIA-first, some are Mac/MLX-first. Make sure the models and apps you want are happy on the platform before you commit.
• These are young products. Firmware and software support are still maturing on the newest boxes, and the reported numbers shift with every driver release. Go in knowing it.

For most tinkerers a good graphics card is still the fast, cheap way in. If you want big models in a small, quiet box, run your models through the calculator first, then have a proper look at the three machine pages - the memory-versus-bandwidth trade lands differently on each.