Mac silicon's LLM speed test: The hardware-software mismatch nobody talks about
The M1 MacBook Air can run a 7-billion-parameter language model at 12 tokens per second. The M4 Ultra hits 70 billion parameters at roughly the same speed. Both facts are true. Neither tells you much about whether you should buy either machine for AI work.
Benchmark culture has infected the Mac silicon narrative like a particularly stubborn virus. Reviewers pit chip against chip, clock raw throughput numbers, and declare winners based on silicon architecture alone. What gets buried beneath the specification sheets is a messier reality: the gap between what Apple's chips can do and what they should do for large language model inference is wider than the marketing wants to admit.
The data exists. It's just rarely presented without spin.
The benchmarking baseline: What the numbers show
Start with the straightforward stuff. An M1 or M2 handles smaller models—the 7-billion-parameter tier—with acceptable latency for interactive use. You'll see 10-15 tokens per second on quantized versions running through MLX or LLaMA.cpp. That's usable. Not fast, but usable.
Jump to the M3 Pro and M3 Max, and the 13-billion-parameter models become practical. The M3 Pro's 12-core CPU and 18-core GPU combo pushes closer to 20-25 tokens per second for quantized inference. Real gains. The M4 Pro improves that further with its 20-core GPU, landing in the 25-30 range depending on model and quantization depth.
Then there's the M4 Ultra. Seventy billion parameters at acceptable latency. The numbers look impressive until you remember that a cloud-based A100 GPU does the same job in one-fifth the time, and the cloud instance costs a fraction of what you'd pay for the hardware.
Here's where it gets interesting: those numbers crumble the moment you change the testing framework. Native implementations using MLX or LLaMA.cpp show one set of results. Generic PyTorch or ONNX implementations running through x86 translation layers show another—typically 40-50% slower. Most benchmarks don't specify which they're using. Guess which one gets highlighted in marketing materials.
An Intel MacBook Air from 2020 can technically run these models too. It'll take five to eight times longer. The fact that it's possible doesn't make it practical.
The silicon gap that marketing glosses over
Apple's unified memory architecture is real. It's a genuine advantage. Separating CPU and GPU memory creates friction; unified memory doesn't. But this advantage has a shelf life measured in minutes, not hours.
Thermal throttling arrives quickly on base M1 and M2 chips during sustained inference workloads. Run a model continuously for 20 minutes and watch performance degrade by 15-20%. The M3 and M4 handle heat better, but they're not magic. This is the conversation nobody has when discussing specs.
The M3 Pro outperforms the base M3 by 15-20% on transformer-heavy workloads. The price jump doesn't match that performance gain. Buyers chasing the "AI Mac" narrative often end up in the wrong configuration—paying M3 Pro prices for M3-base performance.
GPU core count matters more than CPU cores for language models. The M4 Pro's 20-core GPU shows measurable advantages over the M3's variant. But here's the hard truth: 120GB/s is the memory bandwidth ceiling on the fastest Macs. An A100 GPU hits 2TB/s. That's not a gap. That's a chasm. It's why serious inference stays on data centers.
Where the ecosystem actually works
The software layer matters more than people admit. MLX, Apple's in-house framework for machine learning on silicon, closes the performance gap with generic implementations by 10-15% on average. That's not negligible. It's also not enough to change the fundamental calculus.
Quantization—reducing model precision from float32 to int8 or int4—becomes essential below the M2 Pro tier. You lose 1-3% accuracy. You gain 3-4x speed. For prototyping and personal workflows, that trade-off is worth it. For production systems, it's a compromise masquerading as a feature.
Ollama and LM Studio deserve credit for making this accessible. They abstract away the technical friction that would otherwise keep most users away. A developer can now run a 13-billion-parameter model on an M2 MacBook Pro without understanding quantization, memory management, or framework selection. That's a win for adoption.
Enterprise adoption remains minimal. Most teams still default to Linux for reproducibility and cost. The "AI Mac" narrative inflates specs; a $1,200 M3 Pro MacBook Air runs the same 7-billion-parameter models as a $400 used M1, just 30% faster.
The market realities nobody mentions
M1 resale prices still hold at 60-70% of original cost. That's not because they're great for LLM inference. It's because they're great for everything else. The actual LLM performance remains secondary to general usability.
Developers shipping on-device AI—Claude for Mac, Copilot for macOS—have moved to M-series-only requirements. That's a signal. It means the older architecture was too much of a hassle. It doesn't mean the current generation is ideal; it means it's acceptable.
Next year's M5 improvements will matter less than software maturity. MLX optimization still lags behind what mainstream frameworks can do. Ecosystem fragmentation remains the real bottleneck.
What to actually expect going forward
M-series Macs are stabilizing as credible tools for prototyping and fine-tuning, not production inference. That's a respectable niche. Expect an 8-12 month lag before mainstream frameworks optimize for new chip generations. M4 optimization is still incomplete.
Thermal and power management will remain limiting factors. The real play isn't running GPT-4-class models locally. It's reducing API calls through smarter caching and smaller, domain-specific models that M-series hardware can actually handle without breaking a sweat.
That's where the actual value lives—not in the raw speed of the fastest chips, but in the realistic performance of machines that developers can actually afford to own.