Five models on one Mac: what they cost and what fair thermal state takes back

Five models, one machine, the same workload for each: 180 seconds of continuous generation, an identical 566 token prompt, 600 tokens per request, with 120 seconds of idle between models. 82 requests and 126 samples in total.

Three results stand out: what a model actually costs in memory, how little parameter count predicts, and what the fair thermal state takes back.

Method

The machine was a MacBook Pro, Apple M5 Pro with 5 Super and 10 Performance cores, 48 GB, macOS 27.0 build 26A5425a, running Searoom 0.3.0. Models were served by LM Studio through lms load at a 4096 token context and driven through its OpenAI-compatible endpoint on localhost:1234.

Counters came from vm_stat and Searoom --dump-sample, sampled before each model loaded, twice after it loaded, after every request, and through each cooldown.

The first three are one family at three sizes in one format, so across those rungs only the parameter count changes. The last two are deliberate contrasts: Devstral shares the mistral3 architecture but ships as GGUF, and Gemma is the only mixture-of-experts model here.

Idle baseline before anything loaded: 19.91 GB working set, 27.79 °C, thermal state nominal, both fans at 0 RPM.

What a model actually costs

Two memory figures are reported because they answer different questions. After load is what the runtime had materialised when lms load returned. Peak in use is the highest working set observed during generation, which includes the KV cache and activations, and is therefore what the model actually costs you while you are using it.

Resident cost is not file size, and the error is worst at the small end. The 3B model occupied 195% of its file while the 24B occupied 98% of its. That ordering is the opposite of intuitive and it matters: the fixed overhead of a loaded model, the KV cache and activations at a given context length, does not shrink in proportion to the weights. A 2.78 GB file cost 5.43 GB.

The practical consequence is for exactly the people most likely to be counting. Adding up file sizes to decide what fits on a 16 GB Mac will understate a small model by a factor approaching two. Size against measured resident cost at the context length you actually use.

Format explains part of the spread. Devstral at 98% is consistent with llama.cpp memory-mapping GGUF weights, so pages are file-backed and need not all be resident. The MLX models behave differently: they materialise lazily, so what is resident keeps climbing through the first requests rather than settling at load. Sampling only after load would have reported 3.67 GB for the 3B instead of 5.43 GB, understating it by a third.

Parameter count predicts neither speed nor size

Gemma at 26 billion parameters generated 4.22 times faster than Devstral at 24 billion, and occupied 1.14 GB less memory doing it.

The A4B in Gemma's name is the explanation: it is a mixture of experts that activates roughly four billion parameters per token. All 26 billion have to be resident, which is why the memory figures are close, but only a fraction do work on each token, which is why the throughput figures are not. A number in a model's name describes what it holds, not what it runs.

Fair thermal state is not free

The macOS thermal state is usually read as a switch: fine until serious, then throttled. Across five models and 82 requests, that is not what the numbers show. Degradation begins at fair, and on the heaviest workloads it is substantial.

The two heaviest workloads lost roughly a quarter and a third of their peak throughput, entirely inside fair. Neither ever reached serious. Checking the thermal state, seeing fair, and concluding the machine is fine would be wrong by a third.

The 3B model is the control that makes this readable. It ran 31 requests across the same 180 seconds, never left nominal, and was flat: 115.05 tokens per second with a 1.2% spread from peak to last. Time under load by itself does not produce a decline.

Gemma shows the transition most clearly. It held 73.5 tokens per second across fourteen consecutive requests without deviating more than a few hundredths, then began falling on the request where thermal state changed to fair, reaching 54.39 six requests later.

The state itself comes from ProcessInfo's thermalState, which needs no sensor access and is available on every Mac.

ProcessInfo.processInfo.thermalState   # or Searoom --dump-sample
# nominal for 14 requests: 73.80, 73.73, 73.57, 73.57, 73.58, 73.31, 73.51 ...
# fair from request 15:    73.22, 67.55, 64.91, 61.54, 57.74, 54.39

So fair is where degradation begins, not where it is absent. serious is not the onset of throttling; it is a later stage of it.

Limits

One machine, one chassis, one ambient temperature, one context length, one prompt. Five models is enough to show a direction and not enough to fit a curve to. Load times were measured but are not reported, because every model here had been read from disk recently and the figures reflect page cache state rather than the models: the same Devstral build that took 17 seconds from cold in an earlier session loaded in 3.5 seconds here.

Nothing in this run reached serious, so the article still has no measurement of what happens beyond fair. That remains unpublished rather than estimated.

Definitions for every value used here are in the metric reference. The memory mechanics behind the resident-cost figures are in what a second model costs and unified memory, and the diagnostic method this run was built to test is in diagnosing thermal throttling.

Sources

• lms load, LM Studio documentation
• OpenAI-compatible endpoints, LM Studio documentation
• Ministral 3 3B Instruct, model card
• Ministral 3 14B Instruct, model card
• Devstral Small 2 24B Instruct, model card
• ProcessInfo.thermalState, Apple Developer Documentation