Question 1

How much GPU VRAM do I need to run LFM2.5-24B A2B locally?

Accepted Answer

To run LFM2.5-24B A2B (24B parameters) smoothly in Q4_K_M quantization with an 8K context window, we recommend at least 24 GB of dedicated GPU VRAM (after accounting for standard 1.5 GB OS overhead).

Question 2

How do I run LFM2.5-24B A2B in Ollama or vLLM?

Accepted Answer

You can run LFM2.5-24B A2B locally in one command using Ollama: `ollama run liquid-ai-lfm25-24b-a2b` or via vLLM: `vllm serve liquid-ai-lfm25-24b-a2b`.

Question 3

Can I run LFM2.5-24B A2B on Apple Silicon Mac (M1/M2/M3/M4)?

Accepted Answer

Yes. Apple Silicon Macs use Unified Memory shared between CPU and GPU. For LFM2.5-24B A2B, ensure your Mac has at least 32GB to 48GB of Unified Memory, accounting for macOS's ~25% system safety reserve.

Question 4

Does LFM2.5-24B A2B support CPU offloading?

Accepted Answer

Yes, inference runtimes like llama.cpp and Ollama allow offloading transformer layers to system RAM if your GPU VRAM is tight, though full GPU VRAM execution is recommended for peak token generation speed.