Dedicated GPU Setup
Best local text models for 32gb vram
If you are searching for the best text model for 32GB VRAM, this compatibility index displays all open-source Large Language Models (LLMs) that fit your system at a baseline context length of 8,192 tokens. Compare quantization formats (FP16, Q8, Q4) to prevent CUDA OOM crashes.
Supported GPU Configurations & Tiers:
NVIDIA RTX 5000 Ada (32GB)Dual NVIDIA RTX 4080 (32GB)Dual NVIDIA RTX 3080 16GB
Compatible Local LLMs (37)
DeepSeek16B params
DeepSeek V2 Lite
Meta70B params
Llama-3.3-70B
Meta30B params
Muse Glimmer 30B
Meta8B params
Llama-3.1 8B (Instruct)
Meta70B params
Llama-3.1 70B
Alibaba Cloud (Qwen)27B params
Qwen3.6-27B
Alibaba Cloud (Qwen)27B params
Qwen3.8-27B
ornith-ai35B params
Ornith-1.5-35B-A3B
Alibaba Cloud (Qwen)35B params
Qwen3.6-35B-A3B
Alibaba Cloud (Qwen)32B params
Qwen2.5-Coder 32B
Mistral AI7B params
Mistral 7B v0.3
Mistral AI46.7B params
Mixtral 8x7B v0.1
Mistral AI141B params
Mixtral 8x22B v0.1
Mistral AI22B params
Codestral 22B
Mistral AI3B params
Ministral 3 3B (Instruct)
Mistral AI8B params
Ministral 3 8B
Mistral AI14B params
Ministral 3 14B
Mistral AI119B params
Mistral Small 4
Mistral AI128B params
Mistral Medium 3.5
Google2.3B params
Gemma 4 E2B
Google4.5B params
Gemma 4 E4B
Google11.95B params
Gemma 4 12B
Google25.2B params
Gemma 4 26B A4B
Google30.7B params
Gemma 4 31B
Nvidia31.6B params
Nemotron-3 Nano 30B
OpenAI117B params
gpt-oss-120b
OpenAI21B params
gpt-oss-20b
ornith-ai8.2B params
Ornith-1.5-9B
Alibaba Cloud (Qwen)8.2B params
Qwen3.5-9B
Alibaba Cloud (Qwen)3.7B params
Qwen3.5-4B
Alibaba Cloud (Qwen)1.7B params
Qwen3.5-2B
Cloudflare9B params
clef-flash
Bespoke Labs8.21B params
Bespoke-Nimble-9B-v3
Inclusion AI7.9B params
Ling-3.0-tiny
Liquid AI24B params
LFM2.5-24B A2B
Openbmb34B params
MiniCPM V-4.7-35B A3B
Openbmb0.9B params