Is a Mac Actually Good for Running Large Language Models? The Numbers Neither Apple nor NVIDIA Puts on the Product Page

This is for anyone about to buy a machine to run large models locally and deciding from product pages. Most of the data comes from the llama.cpp community’s shared benchmarks; I checked it by running the same command on the Mac mini (M6, 16 GB) and the RTX 3060 laptop I own, and re-ran Apple’s own footnoted test on the 16 GB base model. How I ran everything and the full numbers are in the appendix.

What the product pages say, and what the small print says

The two vendors’ claims next to their footnotes:

Neither vendor says anything false, but both put compute front and centre. The trouble is that the thing you wait for most when running a large model — tokens coming out one by one — barely depends on compute.

Running a model really compares three things

1. How fast it writes: memory bandwidth

A model writes one token at a time. For each token the GPU reads all of the weights out of memory and multiplies them with a tiny bit of fresh state. The multiplying is quick; the reading is slow. So single-user output speed has a rough but useful ceiling:

Output speed ≈ memory bandwidth ÷ bytes the model occupies

llama.cpp has two community benchmark threads (Apple Silicon, NVIDIA) where people post results from the same model and the same command. I first ran that command on my two machines to check: 75.2 on my RTX 3060 laptop against 76.9 for a community 3060 desktop, 35.0 on my Mac mini M6 against 35.7 for a community Mac mini M6. A 2–4 % gap, so the data is trustworthy. Sorted by bandwidth:

It is nearly a straight line. Every machine uses only 56–88 % of its rated bandwidth, but the ranking is set entirely by bandwidth: the M5 Ultra has a fifth more bandwidth than the 4090 and writes at about the same speed; the desktop 5090 has twice the laptop 5090’s bandwidth and writes nearly twice as fast. Both are called 5090; they sit far apart on the line.

2. How fast it reads long input: compute

When the model reads the text you give it (prefill), it processes thousands of tokens at once, and only then does compute become the bottleneck. The gap opens up at once:

The 4090 and the M5 Ultra write at about the same speed, yet read prompts three times apart. You will not notice in a short chat; but ask it to read a code repository, a contract dozens of pages long, or run it as an Agent calling tools turn after turn — rereading an ever-longer context every turn — and three times is three times the wait.

3. How big a model: the memory the GPU can actually use

Bandwidth decides how fast; capacity decides whether at all. A model that does not fit runs at zero. Both vendors print the total; neither prints how much you can actually use:

• A Mac’s GPU cannot use all of the unified memory. On my 16 GB Mac mini, llama.cpp reads a GPU limit of 12.7 GB from the system at start-up, about 79 %, and it shares that with the OS, the browser and your editor.
• An NVIDIA card’s video memory is a hard limit. Whatever does not fit goes back to CPU memory, and speed drops by an order of magnitude (measured below). The largest consumer card, the desktop 5090, has 32 GB.

Apple’s footnoted test, re-run on the base model

Apple’s “13.5x faster” uses a 14B model at 4 bits with an 8K-token prompt and measures time to first token, on a 32 GB machine. I ran the same conditions on a 16 GB Mac mini (MLX, mlx-lm) and added 16K and 32K prompts — Agent work routinely reaches tens of thousands of tokens of context. I also tried the same model on the 3060 laptop with 6 GB of video memory.

• With an 8K prompt the first token takes 15 seconds, then 15 tokens a second. Bearable for everyday questions — and these are the conditions Apple chose.
• 16K takes 35 seconds, and output drops to 13 tokens a second; GPU memory reaches 11.5 GB, just short of the 12.7 GB the system allows.
• 32K crashes: Metal reports the GPU is out of memory and the program exits. A 14B model with 32K of context does not run on the 16 GB base model at all — and an Agent reading a code repository reaches that size routinely.
• The 3060 laptop cannot even hold the model: the 14B needs 8.2 GB, the 6 GB card takes half, the rest runs on the CPU, and output falls to 4.8 tokens a second; all on the CPU it is 2.5. The same card does 75 on a 7B that fits. NVIDIA’s wall is not slowness; past the line it collapses.

The Mac mini is bad at running models, but good at something else

Having covered the weak spots, fairness needs the strengths too. The same 16 GB Mac mini and the RTX 3060 laptop (its CPU is an i7-12700H, a 14-core gaming laptop from 2022) ran the same scripts on the work I actually do every day. The real project is a TypeScript gateway repository with 5,139 tests:

• Writing code, running tests and compiling: the Mac mini is more than twice as fast. About three times faster on one core, seven times faster at moving memory on the CPU; SQLite one-by-one commits are 16 times faster (0.15 s vs 2.4 s).
• Where it is slower is small actions heavy on system calls: spawning subprocesses, creating and deleting small files and git grep are two to three times slower. But these are milliseconds — one subprocess takes 1.4 ms against 0.5 ms, which nobody notices. After 120 seconds at full load it loses about 5 % of its speed; the laptop does not.
• So the line on Apple’s page about using “agents like Claude Code or Codex to be more productive” holds: those Agents run their model in the cloud, and what the local machine does is compile, run tests, read and write files and call git — all things the Mac mini does fast and steadily. Office work and writing are no contest.

One more strength the product page does not even push, and that holds up hard when measured: low power, and quiet. I read whole-machine power from macOS’s own power telemetry (no root needed, sampled every second, averaged from the accumulators), and the 3060 laptop’s GPU power from nvidia-smi:

The Mac mini runs the whole 14B model from memory at under 30 W for the entire machine; the 3060 laptop’s GPU alone idles at just over 30 W and draws 86 W on the same model, while writing only a third as fast because the model does not fit. Even fully loaded, I can barely hear the Mac mini’s fan — my impression, not a decibel measurement. For a computer left on around the clock as an Agent’s workstation, this matters.

The trouble only starts when you move the model onto the same machine.

What Apple does not put up front

1. Bandwidth follows the configuration, and the page says “up to”. The Mac mini M6 page says “up to 170GB/s”; the 16 GB entry model has 153. The Mac Studio page lists 614 GB/s for the M5 Max — that is the 40-core-GPU version; the 32-core entry version has 460. On the line above, a tenth less bandwidth is a tenth slower output.
2. The GPU cannot use all of the unified memory. 16 GB gives the GPU about 12.7 GB, before subtracting whatever else you have open.
3. Reading long input is the weak spot. Every “X times faster” Apple quotes is against older Macs; against an NVIDIA card at the same price, prompt reading is two to three times slower. On the 16 GB base model a 14B with an 8K prompt waits 15 seconds and 32K does not run — and Agents, code repositories and long documents are exactly how “local AI” gets used most.
4. The software version decides whether you get the new hardware. An old MLX cannot use the M6’s Neural Accelerators and reads prompts 3.6 times slower.
5. Its real strength is capacity. A Mac Studio holds up to 128 GB / 512 GB of unified memory; one machine fits a model that would take several NVIDIA cards. On this point the marketing does not exaggerate.

What NVIDIA does not put up front

1. Video memory is a hard limit. However high the AI TOPS on the page, a model that does not fit in video memory falls off a cliff.
2. The laptop and the desktop share a name, not a spec. Laptop 5090: 24 GB, 896 GB/s; desktop 5090: 32 GB, 1,792 GB/s. Same with the 3060: 6 GB on laptops, 12 GB on the desktop card. On a laptop, look at the memory and bandwidth, not the model number.
3. AI TOPS is about reading the prompt. Output speed follows bandwidth, and the desktop 5090’s product page happens not to list it.
4. Power and noise. One desktop 5090 draws 575 W and wants a 1,000 W power supply; a Mac Studio under full load stays far below that.

So is a Mac good for running large models?

China prices for reference (apple.com.cn, 2026-10-18): Mac mini M6 16 GB from ¥6,999; Mac Studio M5 Max from ¥19,999; M5 Ultra from ¥46,999.

In one line: a Mac mini is a very good development and Agent machine, not a good local-LLM machine. Its product page puts the two side by side — “create your own AI agents” and running large models on device — and the first holds up when measured, while the second, on the 16 GB base model, is closer to “it runs, the experience is so-so”. To fit a truly large model locally, look at the large memory of the Mac Studio tier; to run it fast, it is still NVIDIA.

Further up: power and borders

If you keep going — an 8-GPU data-center server, the newest data-center GPU — you hit two ceilings that have nothing to do with performance:

• Power: one NVIDIA DGX Rubin NVL8 draws about 24 kW and needs direct-to-chip liquid cooling; an ordinary socket in a Chinese home is 10 A (about 2.2 kW), and even the 16 A socket for an air conditioner gives about 3.5 kW.
• Borders: the Blackwell and Rubin data-center GPUs are barred from sale to China, with no sign of a licence as of October 2026; the full RTX PRO 6000 (96 GB) is also above the control threshold, and China gets the export-edition RTX PRO 6000D (according to teardowns, 84 GB and about 17 % fewer cores).

Beyond that, you rent GPUs or buy tokens.

Limitations

• The community numbers only cover a 7B model, one user, a 512-token prompt and llama.cpp; entries use different software versions, and newer versions alone are 10–30 % faster, so read the order of magnitude.
• The everyday-development comparison is against a 2022 i7-12700H gaming laptop, not a new PC at the same price; the laptop had more programs open during the run (a quick re-run gave nearly the same single- and multi-core results). Dependency install times depend on the network and caches and are left out.
• Power: the Mac mini numbers come from the system’s own power telemetry, not a wall meter; on the 3060 laptop only the GPU can be read, and CPU power needs root, so it was not measured.
• My re-run covers one 16 GB Mac mini and, on the Mac, one engine (mlx-lm), while Apple’s test used LM Studio; I did not measure the 32 GB version or a Mac Studio. The 14B numbers use Qwen3-14B, which may not be the model in Apple’s footnote.
• Prices and export controls are as checked on 2026-10-18 and change quickly; the 6000D specs come from teardowns and benchmarks, not from NVIDIA.

Sources

• Product pages and footnotes: Apple Mac mini (Chinese edition; footnote 6 gives the conditions for “13.5x”), Mac Studio; NVIDIA RTX 5090, RTX 50 Series laptops. All read on 2026-10-18.
• Benchmarks: llama.cpp community threads Apple Silicon #4167 and NVIDIA #15013 (raw posts fetched 2026-10-17).
• Specs: NVIDIA GPU comparison, RTX 3060 Laptop GPU (Notebookcheck); Apple Mac mini specs, Mac Studio specs; China prices from apple.com.cn.
• Power: Apple Mac mini power consumption and thermal output (M6: 4 W idle, 70 W maximum, measured at the wall); DGX Rubin NVL8; Chinese standard GB 1002-2024.
• Export controls: Tom’s Hardware on the 6000D, PCGH teardown.

Appendix: how I ran it and the full numbers

# The same command as the community benchmark (llama.cpp b11696 official prebuilt binaries)
llama-bench -m llama-2-7b.Q4_0.gguf -ngl 99 -fa 0,1 -p 512 -n 128

# Apple's footnote 6 conditions: 14B, 4 bits, 8K prompt, time to first token (plus 16K and 32K)
mlx_lm.benchmark --model mlx-community/Qwen3-14B-4bit -p 8192 -g 128 -n 2