How much RAM a local model really needs, measured

Your Mac has 16 GB. The model you want is 5 GB on disk. The answer to will it fit is not 5 GB, and it is not 16 − 5 either. Across five models measured under one identical workload, what a model actually occupied while generating ran from 88% to 195% of its file size. The 5 GB file is the least reliable number in the decision.

The direction of the error depends on the model. Small models were the most understated: a 2.78 GB model cost 5.43 GB, 195% of its file. Large ones were the most overstated: a 24B model occupied 98% of its 15.21 GB file, and a 26B mixture-of-experts occupied 88%. Anyone adding up file sizes to decide what fits on a 16 GB Mac is understating a small model by a factor approaching two, and the models people try to fit on 16 GB Macs are small ones.

Where these numbers come from

They are consolidated from one run: five models, one machine, an identical sustained workload for each — five models on one Mac. The machine was a MacBook Pro, Apple M5 Pro, 48 GB, on macOS 27.0 build 26A5425a, counted by Searoom 0.3.0 with vm_stat, with models served by LM Studio at a 4096 token context. Each model ran 180 seconds of continuous generation behind the same 566 token prompt, with counters sampled before load, after load, and after every request.

Two figures, two questions

After load is what the runtime had materialised when the load command returned. Peak in use is the highest working set observed while generating, and it is the one that answers whether the model fits, because it includes what the weights alone do not: the context's key-value cache and the activations of the current request. A model can load and still not leave room for the conversation you opened it for.

The two figures are not close at the small end, and the difference is a trap. After load, the 3B model reported 3.67 GB; at peak it occupied 5.43 GB. A sizing method that samples once, after load, understates that model by a third. The reason is measured in the run: the MLX models materialise lazily, so what is resident keeps climbing through the first requests rather than settling at load.

The working set being counted is defined the same way the metric reference defines it: active plus wired plus compressed pages, capped at physical memory. That is what vm_stat and Activity Monitor's memory graph are reading, combined. The run's idle baseline matters for sizing: this machine idled at 19.91 GB of working set before any model loaded.

Why the small end is the trap

The pattern in the table runs backwards from intuition: peak-vs-file falls as models grow, from 195% down to 88%. The reason is that a loaded model's cost is two parts, and only one of them shrinks with the model. The weights dominate a large model, so its resident cost tracks its file. A small model's weights dominate nothing: the same per-request overhead — cache and activations at a given context length — sits next to 2.78 GB of weights, and that overhead does not shrink in proportion. This is inference from the measured ratios, marked as such: the cache itself is what grows with context, and its growth rate by context length is its own measurement, not yet published.

Format changes the ratio too, and the run measured both behaviours. The GGUF model occupied 98% of its file, consistent with the runtime memory-mapping the weights so pages stay file-backed and need not all be resident at once. The MLX models behaved differently, materialising lazily with residency climbing through early requests. The mapping mechanisms are the run's reasoned attributions; the measured facts are simply that the GGUF model sat at its file size and the small MLX models ran to double it.

One more measured contrast belongs in any sizing decision. The 26B mixture-of-experts model was cheaper in memory than the 24B dense model next to it — 13.76 GB against 14.90 GB — while generating 4.22 times faster. All 26 billion parameters have to be resident, but a number in a model's name describes what it holds, not what it costs.

A sizing method that survives

What replaces the file-size heuristic is a short checklist, each step checked against the run.

1. Budget the working set, not the file. The file is the floor; the measured peak is the cost (88% to 195% of it here).
2. Size against peak in use at the context length you actually use, not against the after-load figure. The after-load figure understated the smallest model here by a third.
3. Expect the ratio to move with model size and format: the smallest MLX model ran to double its file, the GGUF model sat at its file size, and the mixture-of-experts model came in below.
4. Leave headroom for the baseline. This machine idled at 19.91 GB of working set before any model loaded. That baseline scales with total memory, so a 16 GB Mac's idle baseline is smaller — reasoned, not measured here, and precisely why the file-size heuristic fails hardest there: less headroom, bigger relative overhead.
5. Check the pressure, not just the capacity. Fitting is surviving; loading a second model, a browser, and the compressor all compete for the same pool.

Every step is measurable on your own machine, with commands rather than trust. vm_stat prints the components of the working set in pages, sized by sysctl hw.pagesize — 16384 bytes on this machine. Searoom's command line emits one JSON sample on stdout, and primes its own counters first, because a rate has no previous reading to subtract on the very first collection:

sysctl hw.pagesize   # page size, 16384 bytes here
vm_stat              # active/wired/compressed pages of the working set
searoom sample       # one primed JSON sample; running it twice is unnecessary

Load the model, start generating, and read the counters during generation, not after load. The number you get while it works is the number that answers the fit question.

Limits

One machine, one chassis, one context length, one prompt, five models. The 48 GB machine these figures were measured on is not the machine the fit question bites hardest on, and no claim here transfers to a 16 GB or 24 GB Mac unmeasured: what transfers is the direction of the error and the method, not the numbers. Five models show the pattern and cannot fit a curve to it. The key-value cache's share at contexts other than 4096 is not measured here. Load-time figures from the run are excluded deliberately: every model had been read from disk recently, so they reflect page cache state, not the models. Unified memory means the GPU draws from the same pool the weights sit in, which is why the working set is the budget and why nothing on Apple Silicon is free in the way discrete-VRAM intuition suggests.

The definitions behind every value used here are in the metric reference, and the run this consolidates, with its thermal and throughput results, is five models on one Mac.

Sources

• Ministral 3 3B Instruct, model card
• Ministral 3 8B Instruct, model card
• Ministral 3 14B Instruct, model card
• Devstral Small 2 24B Instruct, model card