whichLlmmodel
Back to all articles
User GuidesSeptember 2026•4 min read

How to Find Usable Memory for Local LLMs: Complete OS Guide

Learn how to find your actual usable free RAM and VRAM on Windows, macOS, and Linux—or use the whichllmmodel CLI to inspect it in seconds, to prevent out-of-memory crashes.

ZT
Zubair Tahir
Founder & Lead Developer

1. Why Total Installed Memory Leads to Crashes

When buying hardware, a laptop or PC is advertised with total physical specs—such as 16 GB RAM or an 8 GB VRAM GPU. However, your operating system and background applications constantly consume memory:

  • •Windows / macOS display server & background daemons: Typically consume 3.0 GB to 4.5 GB of system RAM at idle.
  • •Web browsers & developer tools (Chrome, Discord, IDEs, Docker): Often consume another 2 GB to 6 GB.
  • •GPU Display Servers: Windows Desktop Window Manager (DWM) and macOS WindowServer allocate 0.8 GB to 2.0 GB of VRAM just to render your monitors.

If you have a 16 GB laptop and attempt to load a model requiring 14 GB of memory, your operating system will either freeze, kill the process with an Out-Of-Memory (OOM) error, or swap to disk—reducing inference speed from 35 tokens/sec to an unusable 0.5 tokens/sec.

To solve this, whichllmmodel provides two modes:

  1. 1.Total Memory (Default): Enter your physical hardware specs; our engine automatically reserves safe upfront OS buffers (1.5 GB VRAM, 3.0 GB RAM, or 25% on Apple Silicon).
  2. 2.Available Memory: Enter your machine's exact live unallocated memory for a strict 1:1 ceiling without secondary OS deductions.

2. The Fastest Way: Inspect Usable Memory in Seconds via whichllmmodel CLI

If you don't want to dig through OS task managers or terminal monitor tools, you can use the official WhichLLMMODEL CLI utility to detect your machine's hardware and live usable memory in seconds.

Quick Run (Zero Installation)

Run directly in your terminal using npx:

bash

npx whichllmmodel -o

Or Install Globally

bash

npm install -g whichllmmodel
whichllmmodel -o

What the CLI Does in Seconds

  1. 1.Detects Exact Hardware Memory: Automatically identifies and inspects memory numbers based on your hardware type (Dedicated GPU VRAM, Apple Silicon Unified Memory, or System Host RAM).
  2. 2.Outputs Instant Numbers: Displays your physical installed capacity alongside real-time live usable memory right in your terminal.
  3. 3.Auto-Opens Local Finder: When run with -o or --open, it automatically opens your default browser directly into our Local Finder pre-filled with your machine's exact hardware parameters.
Note for Apple Users: On Apple Silicon (M-series), macOS enforces a standard allocation limit where 75% of unified memory is available for GPU/Metal model weights and KV cache, while 25% is reserved for the OS and background display servers to prevent system freezes.

3. How to Find Usable Memory on Windows (Manual)

Checking Usable Host RAM

  1. 1.Press Ctrl + Shift + Esc to open Task Manager.
  2. 2.Click on the Performance tab on the left sidebar, then select Memory.
  3. 3.Look at the bottom of the window for the Available metric (e.g., 7.4 GB Available out of 16.0 GB).
Important: Always use the Available figure, not just "In use", because Windows includes cached standby memory that can be reclaimed when your local model allocates memory.

Checking Usable GPU Dedicated VRAM

  1. 1.In Task Manager, click on the GPU graph under the Performance tab.
  2. 2.Check the Dedicated GPU memory metric (e.g., 1.4 / 8.0 GB).
  3. 3.Subtract the used amount from total to get your usable free VRAM:
Free VRAM = 8.0 GB − 1.4 GB = 6.6 GB
  1. 4.Alternatively, open PowerShell or Command Prompt and run:
powershell

nvidia-smi

Inspect the Memory-Usage column to see current unallocated VRAM.

4. How to Find Usable Memory on macOS

Apple Silicon Macs (M1, M2, M3, M4) share a unified memory architecture across CPU and GPU cores.

Checking Usable Memory via Activity Monitor

  1. 1.Press Cmd + Space, type Activity Monitor, and press Enter.
  2. 2.Select the Memory tab at the top.
  3. 3.Review the footer summary:
  • •Physical Memory: Total hardware (e.g., 18 GB or 36 GB).
  • •Memory Used: Current allocation by macOS and open applications.
  • •Memory Pressure Graph: Confirm that the graph is green (stable).
  1. 4.Calculate usable memory:
Usable Memory = Physical Memory − Memory Used

Checking Usable Memory via Terminal

Open Terminal and run:

bash

vm_stat

or view live metrics with:

bash

top -l 1 | grep PhysMem

5. How to Find Usable Memory on Linux (Manual)

Checking Usable RAM

Open your terminal and run:

bash

free -h

Look at the available column (not the free column):

terminal

              total        used        free      shared  buff/cache   available
Mem:           31Gi       8.2Gi        14Gi       1.1Gi       9.3Gi        22Gi

Linux uses unallocated RAM for filesystem buffers and page caching. The available metric tells you the true memory that local runners (Ollama, llama.cpp, vLLM) can immediately allocate without swapping.

Checking Usable GPU VRAM

For NVIDIA GPUs:

bash

nvidia-smi --query-gpu=memory.total,memory.free --format=csv

For AMD ROCm GPUs:

bash

rocm-smi --showmeminfo vram

6. Summary

Running local LLMs without out-of-memory crashes requires evaluating models against your actual available memory, not raw physical specs:

  1. 1.The Fast Automated Route: Run npx whichllmmodel -o in your terminal to instantly inspect your hardware and launch our Local Finder pre-filled with your exact numbers.
  2. 2.The Manual Route: Always look at real-time free metrics—the Available column in Windows Task Manager and Linux free -h, or Apple Silicon's unallocated unified memory.
  3. 3.Explore with Confidence: Selecting Available Memory mode in whichllmmodel ensures your models and context windows fit comfortably within your hardware budget with zero OOM crashes.
ZT

Written by Zubair Tahir

Founder & Lead Developer

Building independent, empirical evaluation tools to eliminate model decision fatigue for AI engineers and developers.

Interactive Tool

Find the perfect model for your coding workflow

Set your target coding level, reasoning weight, and budget to let our dynamic scoring engine recommend the highest-value model for your stack.

Test Model Sizing in AppCompare Coding Models