Local AI models and their hardware requirements
13 models (10 text · 2 image · 1 video) · 1 engine
- Page updated
Every model and engine RigCheck has a hardware checker for, with what it takes as input, its sizes, whether the weights are out, which runtimes the page covers and its license. Open a model for the memory you need and a checker for your own GPU or Mac.
RigCheck covers 10 language models, 2 image models, 1 video model and 1 engine. Each page gives the files, the memory they need and a checker for your hardware; numbers trace to the developers' own model cards and labeled community reports.
Language models
| Model | First setup with an answer | Weights · License | Runs with |
|---|---|---|---|
| DeepSeek V4.1 Flash | DeepSeek V4.1 Flash September release: Mac with M5 Max, 40-core GPU (128 GB unified): Matches DwarfStar's 128 GB route: Q2 with SSD streaming (September weights) (published route match). DwarfStar runs its Q2 file (365.7 GB) on one 128 GB Mac by streaming routed experts from SSD (--ssd-streaming), with the expert cache sized automatically. DwarfStar publishes no V4.1 speed for a 128 GB Mac (its one Mac figure is Q4 resident on a 512 GB M3 Ultra). Two 128 GB machines can instead hold it resident with tensor parallelism (set machines to two). Image input works on Metal with its separate 0.97 GB vision encoder. DwarfStar is a community engine, not DeepSeek's code; these are its documented setups, not a RigCheck estimate. Keep the GGUF on a fast local SSD (the 188.83 GiB of Engram tables are always read from the file). DeepSeek V4.1 Flash 1017 update: Mac with M5 Max, 40-core GPU (128 GB unified): Experimental: DwarfStar's September setup applied to its 1017 file: Q2 with SSD streaming (1017 weights) (published route match). DwarfStar runs its Q2 file (365.7 GB) on one 128 GB Mac by streaming routed experts from SSD (--ssd-streaming), with the expert cache sized automatically. Two 128 GB machines can instead hold it resident with tensor parallelism (set machines to two). DwarfStar documents and tests these setups with its September file only. On October 19, 2026 it published DeepSeek-V4.1-Flash-1017-IQ2_XXS-Q2_K-imatrix.gguf (365.7 GB, the size of its September Q2; its name gives the same IQ2_XXS / Q2_K mix) and pointed its Q2 download target at it, whose help text names SSD streaming on one 128 GB Mac, tensor parallelism on two, or a larger resident Mac. RigCheck applies the September setup to it. No 1017 speed or 1017 Q4 file is published; its 1017 MXFP4-experts GGUF (501.6 GB) has no documented setup. DwarfStar's vision encoder is from the September checkpoint; it does not say whether it works with the 1017 file. DwarfStar is a community engine, not DeepSeek's code; these setups are documented for its September file, not a RigCheck estimate. Keep the GGUF on a fast local SSD (the 188.83 GiB of Engram tables are always read from the file). Experimental alternative for 1017: Unsloth says its UD-IQ1_S GGUF (233.0 GB; its 110.6 GB Engram table stays on SSD via mmap) works on 128 GB RAM setups such as a MacBook, DGX Spark or RTX Spark, and estimates 128–145 GB of RAM plus VRAM for its 1-bit builds, the top of a 128 GB machine (runtime and context can need more); it runs only in Unsloth's unmerged llama.cpp pull request #30817 branch or Unsloth Desktop, with no published speed. |
Released Sep 10, 2026 | DwarfStar (community; its own Q2 GGUF (365.7 GB) on a fast SSD; SSD streaming on one 128 GB Mac or DGX Spark, documented for the September file (its 1017 file is not yet in its guides); its Q4 is September only), llama.cpp (experimental; an unmerged branch: Unsloth's pull request #30817 for its 1017 GGUF from 233.0 GB (Unsloth: 128 GB RAM setups such as a MacBook, DGX Spark or RTX Spark), or pull request #28696 for mrfakename's 1017 GGUF from 264.5 GB (no hardware named) or your own conversion of the September weights (Q2_K reported on one DGX Spark)), oMLX (community; DeepSeek V4.1 Flash 1017 update only; Apple Silicon Mac; mlx-community oQ builds from 313.3 GB), vLLM / SGLang (community; DeepSeek V4.1 Flash September release only; data-center GPUs: NVIDIA's NVFP4 checkpoint on four GB300; the current 429.0 GB revision needs vLLM pull request #61530, SGLang and stock vLLM load only the previous revision (about 492 GiB)), Ollama (cloud only) |
| Gemma 4 | Gemma 4 E2B: GeForce RTX 3060 (12 GB) + 32 GB RAM: fits: BF16, GPU memory (input embeddings in system RAM) (capacity estimate) Gemma 4 E4B: GeForce RTX 3060 (12 GB) + 32 GB RAM: fits: Q8_0, GPU memory (input embeddings in system RAM) (capacity estimate) Gemma 4 12B: GeForce RTX 3060 (12 GB) + 32 GB RAM: fits: Q5_K_M, GPU memory (input embeddings in system RAM) (capacity estimate) Gemma 4 26B-A4B: GeForce RTX 3060 (12 GB) + 32 GB RAM: fits with caveats: Q8_0, GPU and system RAM (9 of 30 layers on the GPU) (capacity estimate) Gemma 4 31B: GeForce RTX 3060 (12 GB) + 32 GB RAM: fits with caveats: Q6_K, GPU and system RAM (17 of 60 layers on the GPU) (capacity estimate) |
Released | llama.cpp (community), Ollama, llama.cpp (Gemma 4 E2B only; Jetson Orin Nano, NVIDIA's Jetson container) |
| GLM-5.3-Flash | Mac with M5 Max, 40-core GPU (128 GB unified): Matches Unsloth's 3-bit figure (128 GB): UD-IQ3_XXS (120.4 GB) (after raising the Mac's GPU wired-memory limit) (published route match). Unsloth's total-memory table (RAM + VRAM, or unified memory) for its GGUF files with llama.cpp; it does not say at what context length, and which file of a bit width fits at 128 GB is our reading. This setup counts 128 GB of unified memory. Unsloth says 3-bit works on 128 GB devices like a Mac or DGX Spark; the 120.4 GB file leaves little room for the OS and context, and no run or speed is published. Image input adds the 1.2 GB vision file. Use Unsloth's llama.cpp branch (glm5next/upstream) or Unsloth Desktop, as Unsloth's guide says. No published speed for this hardware. On a Mac, raise the GPU wired-memory limit first. | Released | llama.cpp (community; Unsloth's GGUF files and its llama.cpp branch; about 100 GB of RAM + VRAM for 1-bit), KTransformers (community; official FP8 weights, an RTX 40 or 50 card, an AVX-512 CPU, at least 350 GB of system memory), vLLM (server GPUs: NVIDIA Hopper or newer, AMD Instinct MI350X / MI355X), SGLang (server GPUs (H100 to GB300, MI355X)), Ollama (cloud only; glm-5.3-flash:cloud runs on Ollama's servers) |
| Kolibri-1 | No match in this quick-mode sample (official route). Outside this sample, the checker's community route on GeForce RTX 5070 (12 GB) + 128 GB RAM: Matches the community recipe GPU / RAM class; patched runtime required (experimental). The author used Ryzen 7 7800X3D, 128 GB DDR5-6400, 8 threads: CPU only or RTX 5070 attention/shared experts with routed experts in RAM; the page also covers llama.cpp (independent Hob-forge patch, Q4_K_M GGUF); see the checker | Released | vLLM (needs the aleph-alpha-inference plugin, FP8, data-center GPUs), Transformers (transformers 5.20 or newer; the official card gives no separate hardware for this route), llama.cpp (needs a patch; independent Hob-forge patch, Q4_K_M GGUF) |
| LFM2.5-24B-A2B | No match in this quick-mode sample (); see the checker | Released Oct 13, 2026 | llama.cpp, Ollama, MLX, LM Studio, vLLM, SGLang, Transformers |
| Mistral Medium 3.5 | GeForce RTX 3090 / 3090 Ti (24 GB) + 64 GB RAM: fits with caveats: Q3_K_M, GPU and system RAM (22 of 88 layers on the GPU): a dense model runs slowly from RAM (capacity estimate) | Released Apr 28, 2026 | llama.cpp (community), Ollama, vLLM, SGLang, Transformers |
| Nemotron 3.5 | Nemotron 3.5 Bolt 74B-A9B: GeForce RTX 3060 (12 GB) + 32 GB RAM: Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory (published route match). This card is outside the architectures NVIDIA's GGUF card lists (Ada Lovelace, Blackwell), and NVIDIA publishes no recipe for it. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 32 GB. That is Unsloth's planning estimate, not a measured run on this hardware. Nemotron 3.5 Super VL 120B-A12B: GeForce RTX 3090 / 3090 Ti (24 GB) + 64 GB RAM: Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB) (published route match). Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 24 GB of VRAM + 64 GB of RAM = 88 GB. Longer contexts and video need more. |
Released Oct 15, 2026 | llama.cpp (Nemotron 3.5 Bolt 74B-A9B only; NVIDIA's 2-bit GGUF and recipes: a 32 GB+ RTX 40/50 GPU or an RTX Spark; latest llama.cpp built from source), llama.cpp (community; Nemotron 3.5 Super VL 120B-A12B only; ggml-org and Unsloth GGUF files, 64 GB+ of RAM + VRAM or unified memory), vLLM (Nemotron 3.5 Bolt 74B-A9B only; DGX Spark: the 24GB packed build, 256K context, fp8 KV cache, a nightly vLLM image at release), vLLM (Nemotron 3.5 Super VL 120B-A12B only; NVFP4 weights on one DGX Spark, one H200, two H100s or one GB200), Ollama (Nemotron 3.5 Bolt 74B-A9B only; nemotron-3.5-bolt, 25 GB), Ollama (Nemotron 3.5 Super VL 120B-A12B only; nemotron-3.5-super, 80–249 GB), LM Studio (Nemotron 3.5 Super VL 120B-A12B only; at least 70 GB of system memory (LM Studio's listing)) |
| Qwen3.8-27B | GeForce RTX 3060 (12 GB) + 32 GB RAM: fits: UD-IQ2_XXS, GPU memory (input embeddings in system RAM) (capacity estimate) | Released | llama.cpp (community), Ollama, vLLM, SGLang, Transformers |
| Qwen3.8-Flash-Next | GeForce RTX 3090 / 3090 Ti (24 GB) + 64 GB RAM: fits with caveats: UD-IQ1_S, GPU and system RAM (54 GB in RAM) (capacity estimate). Flash-Next uses combined memory totals; per-card layer placement is not modeled. Close fits are uncertain until runtime allocation is checked. | Released | llama.cpp (community), Ollama, LM Studio (community), vLLM, SGLang, Strata (community) |
| Mistral Large 4 | Not derived until the weights are out | Announced Oct 6, 2026; weights expected Oct 31, 2026 (source, read Oct 8, 2026) | Mistral API, Ollama (cloud only) |
Image models
| Model | First setup with an answer | Weights · License | Runs with |
|---|---|---|---|
| FLUX 3 Image Klein | GeForce RTX 3060 (12 GB) + 32 GB RAM: Matches BFL's 8 GB guidance: the int8 files in ComfyUI (published route match). BFL: the int8 ComfyUI files run on 8 GB GPUs such as the RTX 3070. Image model, int8 text encoder and VAE together are 11.0 GB; by our reading, on a card smaller than that ComfyUI keeps them on the GPU one at a time and needs system RAM for the rest. BFL gives no RAM figure. The prompt upsampler BFL says not to skip is a 9B model (6.72 GB w4a8, 9.75 GB int8 or 18.8 GB BF16 in ComfyUI); BFL does not say whether its 8 GB figure includes it. When it does not fit with the image model, run it on its own first (BFL's README) or switch it off in the ComfyUI template, with weaker results. | Released Oct 15, 2026 | ComfyUI (BFL's int8 files on 8 GB+ NVIDIA GPUs; NVFP4 files also published (BFL's own NVFP4 code needs Blackwell)), BFL reference code (CUDA; BF16, FP8 (Ada, Hopper, Blackwell) or NVFP4 (Blackwell), no int8 path), Diffusers (Modular Diffusers pipeline on Diffusers' main branch) |
| Qwen-Image-2.1 | GeForce RTX 3060 (12 GB) + 32 GB RAM: Above the all-components-resident planning budget; offload peak unverified (planning budget). Uncalibrated allowances, not a minimum or a guarantee. Full checkpoint files are counted in host RAM even when mmap or sequential loading can use less. Actual peak depends on attention, prefix caches, VAE tiling, reference sizes and backend. No speed prediction. | Released | ComfyUI (community; Base; Turbo sampling not verified here), Diffusers (Turbo needs Diffusers from GitHub (PR #14950)), stable-diffusion.cpp (community; Base GGUF; Turbo GGUF route not verified here) |
Video models
| Model | First setup with an answer | Weights · License | Runs with |
|---|---|---|---|
| MiniMax H3 | GeForce RTX 3060 (12 GB) + 32 GB RAM: Hardware memory matches a published video recipe; exact clip peak unverified (published recipe match). Matching recipes: SGLang Recipe A (lab): one card, DiT, text encoder and VAE streamed from the checkpoint file on an NVMe drive (Recipe A), BF16; measured at 864×480 (at the native 768P the cookbook says to use video_vae=24 if the decode runs out of memory, not measured); lab run with VRAM capped at 12 GiB and a simulated 32 GB host: shows the memory fits, not your card's speed; a real 32 GB machine re-reads the weights from disk on every step. H3-Base FL2VA, source recipe canvases only. Disk-streaming routes assume an NVMe drive. Host-RAM and lab assumptions remain in the full checker; full 2K needs an API. | Released | SGLang, vLLM-Omni, Diffusers, ComfyUI (community), stable-diffusion.cpp (community) |
Engines
| Model | First setup with an answer | Weights · License | Runs with |
|---|---|---|---|
| Strata checker | GeForce RTX 3060 (12 GB) + 32 GB RAM: meets Strata's requirements, with caveats: IQ2_XS (Strata's published requirements). Your RAM cannot hold this size's experts (about 46 GB of RAM), so Strata switches to low-RAM mode by itself and reads the experts the card does not hold from the SSD while it answers. | Engine (runs the model's own weights) | Strata (Windows / Linux, NVIDIA RTX 20+ or listed AMD, 32 GB of RAM and VRAM together), Strata (experimental; Apple Silicon Mac (Metal, ready-made in the Mac app), macOS 15 or newer), Strata (experimental; older NVIDIA cards such as the GTX 10 series or Tesla P40 / V100 (separate CUDA 12 engine, Windows or Linux)), Strata (experimental; older AMD cards such as the Radeon VII / MI50 (Linux build)), Strata (experimental; Intel Arc (experimental Linux build)) |
"First setup with an answer" is the cheapest setup in a fixed sample (RTX 3060 12 GB, RTX 4060 Ti 16 GB, RTX 3090, RTX 5090, then 64 to 512 GB Macs) where the model's own checker, in the finder's quick mode, gives any answer other than "does not fit" or "not covered", quoted with its caveats and the kind of answer. Quick mode: language models: llama.cpp on one graphics card, a 32K context, no image input (Nemotron 3.5, GLM-5.3-Flash and DeepSeek V4.1 Flash: published recipes and memory tables at their own context); Strata: the better of Windows and Linux, an NVMe SSD; Qwen-Image (a planning budget): base model, INT8, 512×512, batch one, no reference images; Kolibri-1: the official FP8 route at a 32K context; MiniMax H3: local H3-Base FL2VA recipes, assuming an NVMe drive; Nemotron 3.5: Bolt and Super VL on one card with the RAM given; FLUX 3 Image Klein: Black Forest Labs' int8 ComfyUI route on the card. Your exact setup is in the checker. "Runs with" lists the runtimes each page covers: (community) means third-party files or builds, (needs a patch) an independent patch, (cloud only) no local download. Parameters are as each developer states them; the model page says what the count includes. Starting from your hardware instead? See graphics cards or the finder.