What can a Tesla V100 32 GB run locally?
- Page updated
- Answers from each model's own checker
- Card memory from NVIDIA
What runs on a Tesla V100 32 GB? Every answer below is the one RigCheck's own checker for that model gives for this card, with 32 GB or 64 GB of system RAM. Nothing on this page is judged separately.
With 64 GB of system RAM, the checkers have an answer for a Tesla V100 32 GB (32 GB of VRAM) for 8 of the 10 models, sizes and routes they cover: Qwen3.8-Flash-Next in the Strata engine, Qwen3.8-27B in llama.cpp, Gemma 4 31B, Gemma 4 26B-A4B, Gemma 4 12B, Gemma 4 E4B, Gemma 4 E2B, LFM2.5-24B-A2B in llama.cpp with caveats (part in system RAM, an experimental path, a planning budget or an unmeasured peak). With 32 GB, 0 without caveats. The llama.cpp language-model answers are capacity estimates and the Nemotron 3.5, GLM-5.3-Flash and DeepSeek V4.1 Flash answers are published route matches, not measured setups.
Tesla V100 32 GB: model by model
Language models
| Model (checker) | With 32 GB RAM | With 64 GB RAM |
|---|---|---|
| Qwen3.8-Flash-Next in the Strata engine | ! Experimental path IQ2_XS No measured speed for this hardware and setup yet. Runs in low-RAM mode: the 24 GB+ card holds part of the experts and the rest are copied from a file on the SSD into RAM at start. For code, the Coder (IQ1_M) runs here with its experts copied into RAM at start; Strata: it is weaker outside code and in languages other than English. Strata's installer pre-selects IQ3_XXS for 64 GB of RAM + VRAM, which would read experts from the SSD in low-RAM mode here (slower); RigCheck picks IQ2_XS, whose experts Strata copies into RAM at start on this PC. Pascal / Volta cards use Strata's separate CUDA 12 engine package (Windows or Linux), covered by the community "Older GPUs" guide. | ! Experimental path IQ3_S No measured speed for this hardware and setup yet. Fits by the docs' rule (experts + ~10 GB), with little room to spare: close the browser while it runs. Pascal / Volta cards use Strata's separate CUDA 12 engine package (Windows or Linux), covered by the community "Older GPUs" guide. |
| Qwen3.8-27B in llama.cpp | ! Fits with caveats (capacity estimate) UD-Q4_K_XL · system RAM, CPU only No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. | ! Fits with caveats (capacity estimate) Q8_0 · system RAM, CPU only No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. |
| Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B) in llama.cpp | ! Best size: fits with caveats (capacity estimate) Gemma 4 31B: fits with caveats (capacity estimate) (IQ4_XS, system RAM, CPU only); Gemma 4 26B-A4B: fits with caveats (capacity estimate) (UD-Q4_K_XL, system RAM, CPU only); Gemma 4 12B: fits with caveats (capacity estimate) (Q8_0, system RAM, CPU only); Gemma 4 E4B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 E2B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only) No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. | ! Best size: fits with caveats (capacity estimate) Gemma 4 31B: fits with caveats (capacity estimate) (Q8_0, system RAM, CPU only); Gemma 4 26B-A4B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 12B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 E4B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 E2B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only) No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. |
| LFM2.5-24B-A2B in llama.cpp | ! Fits with caveats (capacity estimate) Q6_K · system RAM, CPU only No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. | ! Fits with caveats (capacity estimate) BF16 · system RAM, CPU only No measured speed for this hardware and setup yet. RigCheck capacity estimate, not a measured setup. |
No answer here: Qwen3.8-Flash-Next in llama.cpp, Mistral Medium 3.5 in llama.cpp, Mistral Large 4, Kolibri-1 (official FP8 route), Nemotron 3.5 (Bolt 74B-A9B, Super VL 120B-A12B), GLM-5.3-Flash (Unsloth GGUF), DeepSeek V4.1 Flash (September release, 1017 update)
| Model (checker) | With 32 GB RAM | With 64 GB RAM |
|---|---|---|
| Qwen3.8-Flash-Next in llama.cpp | ✕ Does not fit Even the smallest file (UD-IQ1_S, 72.5 GB) needs about 76.1 GB in system RAM. This checker does not cover your graphics card, so it is not counted; its memory could change the answer. | ✕ Does not fit Even the smallest file (UD-IQ1_S, 72.5 GB) needs about 76.1 GB in system RAM. This checker does not cover your graphics card, so it is not counted; its memory could change the answer. |
| Mistral Medium 3.5 in llama.cpp | – Not covered | – Not covered |
| Mistral Large 4 | … Weights not released smallest estimate ~210 GB vs 64 GB of memory (VRAM + RAM, or a Mac's unified memory) | … Weights not released smallest estimate ~210 GB vs 96 GB of memory (VRAM + RAM, or a Mac's unified memory) |
| Kolibri-1 (official FP8 route) | – Not covered | – Not covered |
| Nemotron 3.5 (Bolt 74B-A9B, Super VL 120B-A12B) | – Not covered | – Not covered |
| GLM-5.3-Flash (Unsloth GGUF) | – Not covered | – Not covered |
| DeepSeek V4.1 Flash (September release, 1017 update) | – Not covered | – Not covered |
Image models
No answer in this setup: Qwen-Image-2.1 (image generation), FLUX 3 Image Klein (int8, ComfyUI)
| Model (checker) | With 32 GB RAM | With 64 GB RAM |
|---|---|---|
| Qwen-Image-2.1 (image generation) | – Not covered | – Not covered |
| FLUX 3 Image Klein (int8, ComfyUI) | – Not covered | – Not covered |
Video models
No answer in this setup: MiniMax H3 (video + audio)
| Model (checker) | With 32 GB RAM | With 64 GB RAM |
|---|---|---|
| MiniMax H3 (video + audio) | – Not covered | – Not covered |
Quick setup, as in the home page finder: one Tesla V100 32 GB, a 32K context and no image input for the llama.cpp capacity estimates (Nemotron 3.5, GLM-5.3-Flash and DeepSeek V4.1 Flash are matched against published recipes, memory tables and routes at their own context instead (NVIDIA's 64K / 256K recipes; Unsloth's GLM table states no context length; DwarfStar's routes per DeepSeek checkpoint (documented on Macs and the DGX Spark, experimental on Strix Halo) and, for 1017 only, Strata's route on an RTX 30/40/50 card with 12 GB or more or a Radeon AI PRO R9700 (the selected RAM at least 46 GB, an NVMe SSD, 8K context, text only), not llama.cpp).), the better of Windows and Linux for Strata, an NVMe SSD; for the image and video rows, Qwen-Image (a planning budget): base model, INT8, 512×512, batch one, no reference images; Kolibri-1: the official FP8 route at a 32K context; MiniMax H3: local H3-Base FL2VA recipes, assuming an NVMe drive; Nemotron 3.5: Bolt and Super VL on one card with the RAM given; FLUX 3 Image Klein: Black Forest Labs' int8 ComfyUI route on the card. "With caveats" means part of the model sits in system RAM (slower), an unmeasured peak, or a route the checker marks conditional. Speeds appear only where they were measured on this very card in this setup; official estimates and runs on other setups are left out (the full checkers show them, labeled). Pick this card in a checker for your exact setup.
What more RAM would change
- Qwen3.8-Flash-Next in llama.cpp: with 96 GB of RAM, fits with caveats (capacity estimate) (UD-IQ3_XXS, system RAM, CPU only).
Measured a local model on your own hardware, or tried and it failed? Send us the numbers: paste the benchmark output or log into the form (or email it). After checking it we add it to this page, labeled as a community report and shared under CC BY 4.0.
About the Tesla V100 32 GB
Tesla V100 32 GB: 32 GB HBM2 of video memory (NVIDIA Tesla V100 datasheet (PDF)). For the large mixture-of-experts models here, system RAM matters as much as VRAM: the parts that do not fit on the card are kept in RAM.
This page is the 32 GB Tesla V100, a data-center card (memory read from NVIDIA's datasheet on October 26, 2026). NVIDIA also sold a 16 GB V100; its answers are on the 16 GB VRAM page.
FAQ
Can a Tesla V100 32 GB run Qwen3.8-Flash-Next in the Strata engine?
Only on an experimental path, with 64 GB of system RAM in RigCheck's quick setup: IQ3_S; No measured speed for this hardware and setup yet; Fits by the docs' rule (experts + ~10 GB), with little room to spare: close the browser while it runs. Pascal / Volta cards use Strata's separate CUDA 12 engine package (Windows or Linux), covered by the community "Older GPUs" guide. The full checker takes your exact setup (RAM, disk, operating system, processor) and gives the install steps.
Can a Tesla V100 32 GB run Qwen3.8-27B in llama.cpp?
With caveats, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: Q8_0 · system RAM, CPU only; No measured speed for this hardware and setup yet; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM, image input) and gives the command to start it.
Can a Tesla V100 32 GB run Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B) in llama.cpp?
With caveats, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: Gemma 4 31B: fits with caveats (capacity estimate) (Q8_0, system RAM, CPU only); Gemma 4 26B-A4B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 12B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 E4B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); Gemma 4 E2B: fits with caveats (capacity estimate) (BF16, system RAM, CPU only); No measured speed for this hardware and setup yet; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM, image input) and gives the command to start it.
Can a Tesla V100 32 GB run LFM2.5-24B-A2B in llama.cpp?
With caveats, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: BF16 · system RAM, CPU only; No measured speed for this hardware and setup yet; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM) and gives the command to start it.
Other graphics cards
RTX 5090 · RTX 3090 · RTX 4090 · RTX 4060 Ti 16 GB · Radeon VII / Instinct MI50 16 GB · RTX 3060 12 GB · Every 32 GB card · All graphics cards · Any other card or Mac
Sources
- NVIDIA Tesla V100 datasheet (PDF): memory size
- DeepSeek V4.1 Flash (September release, 1017 update): the checker, its sources and assumptions
- FLUX 3 Image Klein (int8, ComfyUI): the checker, its sources and assumptions
- Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B) in llama.cpp: the checker, its sources and assumptions
- GLM-5.3-Flash (Unsloth GGUF): the checker, its sources and assumptions
- Kolibri-1 (official FP8 route): the checker, its sources and assumptions
- LFM2.5-24B-A2B in llama.cpp: the checker, its sources and assumptions
- MiniMax H3 (video + audio): the checker, its sources and assumptions
- Mistral Large 4: the checker, its sources and assumptions
- Nemotron 3.5 (Bolt 74B-A9B, Super VL 120B-A12B): the checker, its sources and assumptions
- Qwen3.8-27B in llama.cpp: the checker, its sources and assumptions
- Qwen3.8-Flash-Next in llama.cpp: the checker, its sources and assumptions
- Qwen-Image-2.1 (image generation): the checker, its sources and assumptions
- Qwen3.8-Flash-Next in the Strata engine: the checker, its sources and assumptions