Notes on coding using locally hosted models with an M4 Pro Mac mini

TL;DR:

  • Memory bandwidth matters as much as total unified memory; you can load larger models into RAM than you can effectively use on an M4 Pro Mac mini with 64 GB. To fully use that amount, you’d need the M4 Max or M3 Ultra. If buying new, look to the M5 Max or M5 Ultra.
  • Bigger isn’t always better in terms of model parameter count and quants. Larger quantizations may not be tuned well; the QAT models are tuned for their lower quantized size; MoE (mixture-of-experts) models can be very effective with just 3B active parameters.
  • The harness matters. Using Claude, I got Qwen 3 to perform flawlessly, but with “pi” it struggled. I could repeat this same test with OpenCode or Codex or others, and I’d never finish. Finding the best combination is hard.

Models tested: Gemma 4 variants, Qwen 3.x variants, Muse Glimmer

Hardware: M4 Pro Mac mini with 64 GB RAM

Model harness: “pi”

Model server: llama-server from llama.cpp

The test prompt: “Build a Hammurabi game in Go.”

I allowed one review round: “Review the code.”

This is a useful test since all the models should be familiar with the 1968 PL/I version or the 1973 Creative Computing magazine BASIC game. The model has to translate the rules to Go, then it may, if it’s doing well, test the gameplay until it works. We give it one review pass to catch mistakes.

The task shows: (1) Given a straightforward assignment, can the model competently write and test code? (2) Can it avoid basic logic errors? (3) Will it exercise any creativity? (4) How fast does it run?

Baseline

The best online models (Claude Fable 5, GPT-5.6) can do a nearly perfect job in one shot with no struggle. Claude made one mistake, found upon review; it also corrected at least two bugs it knew of from the 1973 BASIC version. GPT-5.6 found only one small issue and made no changes at all during review. Both games play fine, but the Claude version retains the richer dialog from the original and adjusts the grain market every turn.

“pi” Coding Harness: Qwen 3.8 27B Q8 Unsloth Quant with llama-server

The system easily generated something game-shaped all at once, then played until it got a reasonable-looking playthrough. It struggled to produce a correct version. The review found many elementary mistakes, and fixing them resulted in probably five times as much work as the initial prompt. It thrashed and struggled to make correct edits. It finally succeeded after about an hour. The game is missing a few elements that would make it challenging, like rats and bad weather.

The model takes in about 125 tokens/s and outputs about 8 tokens/s.

Model: Gemma 4 26B MoE Q4 QAT

The agent ran much faster (I have the M4 Pro, so larger dense models like Qwen 3.8 27B are bottlenecked by memory throughput). It generated better code on the first try than Qwen 3.8, though the review caught many small issues. Like Qwen 3.8, it had some trouble with tool use and editing code – this could be the “pi” harness, not the model.

The actual game is the best yet, even compared to Claude and GPT-5.6. It didn’t reproduce the original game exactly, but was more creative with the wording and made it easier to play (an “auto-ration” option for feeding the people!). It forgot to allow for buying or selling land, however, which limits what you can do once the population changes a lot. Migration happens, unlike in some other Gemma 4-produced games. In real-world use, I’d just prompt it to add this, and we’d have the best game of all my tests. It ran very fast at around 55 output tokens/s and 700 input tokens/s.

Model: Gemma 4 26B MoE Q8

This one had very promising reasoning output and initial code generation. It got stuck in an infinite loop testing the game. That could be the “pi” harness more than the model. The model ran nearly as fast as the Q4 QAT variant but seemed to perform slightly worse.

While the code quality looked good, the gameplay was nearly broken because it didn’t properly calibrate how much grain the population would consume. It did automatically include immigration and land purchasing.

Model: Qwen 3 Coder 30B MoE Q6

This one runs about as fast as the Gemma 4 MoE models. It too got stuck in an infinite loop. Ignoring that, it produced decent-looking code, better organized than what most other models made. However, it had a fatal flaw in the game where most people starved even when you fed them enough.

It included all the elements of the game, including land, immigration, and pests. Given another chance to review, it fixed the issue and produced an almost flawless game. While fixing it, the model engaged in a lot of circular thinking, returning to the same idea over and over, more than I observed with other models.

Model: Qwen 3.6 35B MoE Q6

The model runs fast on my hardware, just a bit slower than the Gemma 4 26B MoE models at 4- or 6-bit quants.

This was only one of two local models to report knowing about the existence of the 1968 game. Therefore, I can’t fault it for less creativity; it pretty faithfully recreated a recognizable version of the game with some improvements.

Instead of getting stuck in a loop during testing, it diagnosed the issue and fixed it, with some struggle. As it did this, it reread the code and corrected a number of other code-quality and logic problems. The review skills it showed are strong. It’s the only model that produced a bit of documentation.

The code quality is good. Its code is the largest of all the versions of the game, but it includes more error-checking and input validation, so that’s fair. It made a proper go.mod file and named the binary instead of just “main.” Most other local models didn’t add any code comments. Qwen 3.6 added helpful comments to the code. LLMs have a tendency to add too much useless commentary, so “no comments” can be an improvement, but in this case the comments made reading the code for the first time much easier.

While not as creative as Gemma 4 31B, it generated good dialog and in-game directions.

Model: Gemma 4 31B Q4 QAT

It runs fast compared to the Qwen 3.8 27B Q8 dense model, but slowly compared to the 26B MoE Q4 QAT – we’re just over the memory bottleneck. It produces something like 16 output tokens/s and 150 input tokens/s. It’s just fast enough to be acceptable, I feel.

Two “pi” quirks: First, it built a version of the game and tested it interactively (it can do this), but introduced a bug where the input prompt went into an infinite loop. This was not necessarily caused by “pi,” but Claude Code or Codex would have handled the result better. Next, when the model finished, it didn’t provide any signal that it was actually done; it just spit out all the code to “pi” and the output file. This is probably a mismatch between “pi” and the Gemma 4 models.

That said, the creative choices were pretty good, arguably better even than the Gemma 4 26B MoE Q4 QAT. It didn’t include immigration to the city. It didn’t rank performance well, though: I starved 90% of my people, and it said I was a well-regarded ruler, which is not how it’s supposed to rank such a game. Instead, it should say something like, “87 people would like to see you dead, but then again, we all have our trivial little problems.”

It made the most coding mistakes – no Go module and syntax errors in the second pass (the first built fine).

Model: Gemma 4 31B Q6

This one runs slightly slower than the Q4 QAT quantization, as predicted. It appeared to produce substantially better-quality code and reasoning than any other model tested, while pushing the Mac mini harder than any other workload in these tests so far. You can see the reasoning trace with “pi,” so I was able to note that this version of Gemma 4 seemed to think along lines closer to the goal and raised sensible doubts while getting less confused than other models.

It had a better understanding of the task, made more creative choices (adding gold coins to the economic model), and made no mistakes while describing what it had done and its progress in its work. Even before the review, the game worked perfectly well, except it didn’t think to include migration to the city, much like the other Gemma 4 31B model. During the review, it noticed it had forgotten to make a land-purchase option while adding migration.

For creative problem-solving, software design, and overall “good ideas,” Gemma 4 31B Q6 ranks the highest.

Model: Meta’s Muse Glimmer 30B Q8

It runs a bit slower than the Gemma 4 31B Q6, but comparably – bottlenecked by memory bandwidth. It got stuck testing the game in an I/O loop (this must be a harness failure). Once alerted, it debugged the issue perfectly, however, unlike a couple of others. The internal reasoning dialog is much more terse and to the point than Gemma or Qwen. Muse was aware of the 1968 PL/I game.

It produced some test files and a bit of documentation. The code is direct and to the point. The gameplay is similar; nothing is missing, but it didn’t add any interesting touches except a score at the end. Muse made a working solution on the first shot with one minor bug and the I/O issue (only a problem for testing, not users). The review didn’t turn up much to correct.

The main thing to know is that Muse Glimmer behaved most like a real pro-level coding agent, producing less questionable code and heading in the right direction immediately. It’s surprising since it’s not exclusively a code-only model; it’s for “agentic always-on” use, multimodal and multilingual. They claim good tool use and “LLM as judge” evaluation. Based on what I saw, I can believe it – Muse recovered better than the other models from problems using “pi.” To get effective use out of it, I’d want an M3 Ultra or M4 Max and probably 96 GB.

General Notes

A pattern I’m picking up on seems to be that there’s a balance between decision-making and pattern-following on one hand, and real-world knowledge on the other. Gemma 4 doesn’t seem to know the details of the Hammurabi game, but it has ideas for how one might look, which Claude and GPT-5.6 really didn’t – they just reproduced the original.

Regarding code quality, it’s tough to judge. The best code may have actually been made by the Qwen 3.8 27B Q8 model, even though the model appeared to struggle (partly it could be due to the harness). The struggle may actually be positive: It noticed more issues, so it kept trying to resolve them during review. The actual game wasn’t great, however, so I’m not sure.

The Qwen 3 Coder model struggled to make a playable game, though the code looked superficially decent. The Qwen 3.6 35B MoE Q6 variant did very well, on the other hand, so that’s the Qwen model I’d choose until a 3.8 version gets made that works better – I think it may be that there are bugs in the Qwen 3.8 GGUF file or that it functions uniquely poorly with “pi.”

The Gemma 4 26B MoE Q4 QAT did a pretty good job, making decent abstractions and functions, probably better than the other Gemma 4 models. Both this model and the Qwen model (correctly, I feel) made a go.mod file along with the main.go file. This makes building with the go tool work as expected. The other models just made a bare main.go, which works, but is kind of below expectations. Remember, though, this was with no prompting at all on project organization or style.

The Gemma 4 31B Q6 model output was quite reasonable in terms of style, just maybe not the best.

Overall, I’d say that if you have the hardware I do, Gemma 4 31B Q6 or a slightly lower quantization would be the best overall choice for local coding and general use – Qwen seems to struggle outside of pure coding while performing only marginally better, if at all, on code. If you have better hardware, I still think Gemma 4 is a strong contender.

If you’re constrained to a small video card or under 32 GB of RAM on a Mac, go with the Gemma 4 26B MoE QAT model at the highest quantization level offered. It’s faster and almost as good as the dense Gemma 4. The Qwen 3.6 35B MoE Q6 may be the most competent purely for software development (no creativity or general knowledge), so if that’s all you need, pick the highest-quant Qwen 3.6 MoE that fits on your system while leaving room for 64K tokens of context.

The Muse Glimmer 30B model is intriguing. I could only try the 8-bit quant, but at that size it got pretty bogged down; maybe a lower quant would be worthwhile. If I had an M4 Max, I think it would be my first choice at 8-bit or FP16 if I had 96 or 128 GB. The vision capabilities are really interesting to me.

Memory bandwidth really matters: My Mac tops out at 270 GB/s, whereas an M4 Max offers double that. I should have gotten a MacBook Pro or Mac Studio when it was more affordable to get the Max series. An M3 Ultra (only in the old Mac Studio, I think) has even more bandwidth. So if you’re buying used, look for a Max or Ultra with more than 64 GB, or if buying new, look for an M5 Max or Ultra.