Skip to content
RigCheckWhat can I run?

What can an RTX 4060 Ti 16 GB run locally?

What runs on an RTX 4060 Ti 16 GB? Every answer below is the one RigCheck's own checker for that model gives for this card, with 32 GB or 64 GB of system RAM. Nothing on this page is judged separately.

With 64 GB of system RAM, the checkers have an answer for an RTX 4060 Ti 16 GB (16 GB of VRAM) for 12 of the 15 models, sizes and routes they cover: 7 without caveats (Qwen3.8-Flash-Next in the Strata engine, Qwen3.8-27B in llama.cpp, Gemma 4 31B, Gemma 4 26B-A4B, Gemma 4 12B, Gemma 4 E4B, Gemma 4 E2B); Mistral Medium 3.5 in llama.cpp, MiniMax H3 (video + audio), Nemotron 3.5 Bolt 74B-A9B, Nemotron 3.5 Super VL 120B-A12B, FLUX 3 Image Klein (int8, ComfyUI) with caveats (part in system RAM, an experimental path, a planning budget or an unmeasured peak). With 32 GB, 6 without caveats (Qwen3.8-27B in llama.cpp, Gemma 4 31B, Gemma 4 26B-A4B, Gemma 4 12B, Gemma 4 E4B, Gemma 4 E2B). The llama.cpp language-model answers are capacity estimates and the Nemotron 3.5, GLM-5.3-Flash and DeepSeek V4.1 Flash answers are published route matches, not measured setups.

RTX 4060 Ti 16 GB: model by model

Language models

Model (checker)With 32 GB RAMWith 64 GB RAM
Qwen3.8-Flash-Next in the Strata engine! Runs, with caveats
IQ3_XXS
No measured speed for this hardware and setup yet.
Your RAM cannot hold this size's experts (about 53 GB of RAM), so Strata switches to low-RAM mode by itself and reads the experts the card does not hold from the SSD while it answers. Strata: slower, and much slower with a small graphics card; RigCheck has no matching speed for this. For code, the Coder (IQ1_M) runs here with its experts copied into RAM at start; Strata: it is weaker outside code and in languages other than English.		✓ Runs
IQ3_S
No measured speed for this hardware and setup yet.
Fits by the docs' rule (experts + ~10 GB), with little room to spare: close the browser while it runs.
Mistral Medium 3.5 in llama.cpp✕ Does not fit
Even the smallest file (UD-IQ2_XXS, 34.9 GB) needs about 49 GB.		! Fits with caveats (capacity estimate)
UD-IQ3_XXS · GPU and system RAM (16 of 88 layers on the GPU): a dense model runs slowly from RAM
No measured speed for this hardware and setup yet.
RigCheck capacity estimate, not a measured setup.
Qwen3.8-27B in llama.cpp✓ Fits (capacity estimate)
UD-IQ3_XXS · GPU memory (input embeddings in system RAM)
No official or community speed for this setup; RigCheck Lab measured Qwen3.8-27B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below.
RigCheck capacity estimate, not a measured setup.		✓ Fits (capacity estimate)
UD-IQ3_XXS · GPU memory (input embeddings in system RAM)
No official or community speed for this setup; RigCheck Lab measured Qwen3.8-27B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below.
RigCheck capacity estimate, not a measured setup.
Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B) in llama.cpp✓ Best size: fits (capacity estimate)
Gemma 4 31B: fits (capacity estimate) (UD-IQ2_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 26B-A4B: fits (capacity estimate) (UD-IQ3_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 12B: fits (capacity estimate) (Q6_K, GPU memory (input embeddings in system RAM)); Gemma 4 E4B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM)); Gemma 4 E2B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM))
No official or community speed for this setup; RigCheck Lab measured Gemma 4 26B-A4B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below.
RigCheck capacity estimate, not a measured setup.		✓ Best size: fits (capacity estimate)
Gemma 4 31B: fits (capacity estimate) (UD-IQ2_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 26B-A4B: fits (capacity estimate) (UD-IQ3_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 12B: fits (capacity estimate) (Q6_K, GPU memory (input embeddings in system RAM)); Gemma 4 E4B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM)); Gemma 4 E2B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM))
No official or community speed for this setup; RigCheck Lab measured Gemma 4 26B-A4B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below.
RigCheck capacity estimate, not a measured setup.
Nemotron 3.5 (Bolt 74B-A9B, Super VL 120B-A12B)! Best size: a published route or recipe matches, with caveats
Nemotron 3.5 Bolt 74B-A9B: a published route or recipe matches, with caveats (Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory); Nemotron 3.5 Super VL 120B-A12B: does not fit
Nemotron 3.5 Bolt 74B-A9B: NVIDIA publishes no recipe for a 16 GB card; its discrete-GPU recipes start at 32 GB. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 32 GB. That is Unsloth's planning estimate, not a measured run on this hardware. Nemotron 3.5 Super VL 120B-A12B: Unsloth plans at least 64 GB of RAM or unified memory for its smallest GGUF (UD-IQ1_M, 46.1 GB); llama.cpp can split the model across RAM and VRAM. This setup counts 16 GB of VRAM + 32 GB of RAM = 48 GB.		! Best size: a published route or recipe matches, with caveats
Nemotron 3.5 Bolt 74B-A9B: a published route or recipe matches, with caveats (Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory); Nemotron 3.5 Super VL 120B-A12B: a published route or recipe matches, with caveats (Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB))
Nemotron 3.5 Bolt 74B-A9B: NVIDIA publishes no recipe for a 16 GB card; its discrete-GPU recipes start at 32 GB. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 64 GB. That is Unsloth's planning estimate, not a measured run on this hardware. Nemotron 3.5 Super VL 120B-A12B: Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 16 GB of VRAM + 64 GB of RAM = 80 GB. Longer contexts and video need more.
No answer here: Qwen3.8-Flash-Next in llama.cpp, Mistral Large 4, Kolibri-1 (official FP8 route), GLM-5.3-Flash (Unsloth GGUF), DeepSeek V4.1 Flash (September release, 1017 update) in DwarfStar
Model (checker)With 32 GB RAMWith 64 GB RAM
Qwen3.8-Flash-Next in llama.cpp✕ Does not fit
Even the smallest file (UD-IQ1_S, 72.5 GB) needs about 76.1 GB.		✕ Does not fit
Even the smallest file (UD-IQ1_S, 72.5 GB) needs about 76.1 GB.
Mistral Large 4… Weights not released
smallest estimate ~210 GB vs 48 GB of memory (VRAM + RAM, or a Mac's unified memory)		… Weights not released
smallest estimate ~210 GB vs 80 GB of memory (VRAM + RAM, or a Mac's unified memory)
Kolibri-1 (official FP8 route)– Not covered– Not covered
GLM-5.3-Flash (Unsloth GGUF)✕ Does not fit
Unsloth's table puts the 1-bit GGUF (93.1–97.6 GB) at 100 GB of total memory, RAM + VRAM or unified memory. This setup counts 16 GB of VRAM + 32 GB of RAM = 48 GB.		✕ Does not fit
Unsloth's table puts the 1-bit GGUF (93.1–97.6 GB) at 100 GB of total memory, RAM + VRAM or unified memory. This setup counts 16 GB of VRAM + 64 GB of RAM = 80 GB.
DeepSeek V4.1 Flash (September release, 1017 update) in DwarfStar– Not covered– Not covered

Image models

Model (checker)With 32 GB RAMWith 64 GB RAM
FLUX 3 Image Klein (int8, ComfyUI)! A published route or recipe matches, with caveats
Matches BFL's 8 GB guidance: the int8 files in ComfyUI
BFL: the int8 ComfyUI files run on 8 GB GPUs such as the RTX 3070. Image model, int8 text encoder and VAE together are 11.0 GB; by our reading, on a card smaller than that ComfyUI keeps them on the GPU one at a time and needs system RAM for the rest. BFL gives no RAM figure. The prompt upsampler BFL says not to skip is a 9B model (6.72 GB w4a8, 9.75 GB int8 or 18.8 GB BF16 in ComfyUI); BFL does not say whether its 8 GB figure includes it. When it does not fit with the image model, run it on its own first (BFL's README) or switch it off in the ComfyUI template, with weaker results.		! A published route or recipe matches, with caveats
Matches BFL's 8 GB guidance: the int8 files in ComfyUI
BFL: the int8 ComfyUI files run on 8 GB GPUs such as the RTX 3070. Image model, int8 text encoder and VAE together are 11.0 GB; by our reading, on a card smaller than that ComfyUI keeps them on the GPU one at a time and needs system RAM for the rest. BFL gives no RAM figure. The prompt upsampler BFL says not to skip is a 9B model (6.72 GB w4a8, 9.75 GB int8 or 18.8 GB BF16 in ComfyUI); BFL does not say whether its 8 GB figure includes it. When it does not fit with the image model, run it on its own first (BFL's README) or switch it off in the ComfyUI template, with weaker results.
No answer here: Qwen-Image-2.1 (image generation)
Model (checker)With 32 GB RAMWith 64 GB RAM
Qwen-Image-2.1 (image generation)– Not covered– Not covered

Video models

Model (checker)With 32 GB RAMWith 64 GB RAM
MiniMax H3 (video + audio)! A published route or recipe matches, with caveats
Hardware memory matches a published video recipe; exact clip peak unverified
H3-Base FL2VA, source recipe canvases only. Disk-streaming routes assume an NVMe drive. Host-RAM and lab assumptions remain in the full checker; full 2K needs an API.
Matching recipes: SGLang Recipe A (lab): one card, DiT, text encoder and VAE streamed from the checkpoint file on an NVMe drive (Recipe A), BF16; measured at 864×480 (at the native 768P the cookbook says to use video_vae=24 if the decode runs out of memory, not measured); lab run with VRAM capped at 12 GiB and a simulated 32 GB host: shows the memory fits, not your card's speed; a real 32 GB machine re-reads the weights from disk on every step		! A published route or recipe matches, with caveats
Hardware memory matches a published video recipe; exact clip peak unverified
H3-Base FL2VA, source recipe canvases only. Disk-streaming routes assume an NVMe drive. Host-RAM and lab assumptions remain in the full checker; full 2K needs an API.
Matching recipes: SGLang Recipe A (lab): one card, DiT, text encoder and VAE streamed from the checkpoint file on an NVMe drive (Recipe A), BF16; measured at 864×480 (at the native 768P the cookbook says to use video_vae=24 if the decode runs out of memory, not measured); lab run with VRAM capped at 12 GiB and a simulated 32 GB host: shows the memory fits, not your card's speed; a real 32 GB machine re-reads the weights from disk on every step

Quick setup, as in the home page finder: one RTX 4060 Ti 16 GB, a 32K context and no image input for the llama.cpp capacity estimates (Nemotron 3.5, GLM-5.3-Flash and DeepSeek V4.1 Flash are matched against published recipes, memory tables and routes at their own context instead (NVIDIA's 64K / 256K recipes; Unsloth's GLM table states no context length; DwarfStar's routes per DeepSeek checkpoint (documented for the September weights, experimental for 1017), not llama.cpp).), the better of Windows and Linux for Strata, an NVMe SSD; for the image and video rows, Qwen-Image (a planning budget): base model, INT8, 512×512, batch one, no reference images; Kolibri-1: the official FP8 route at a 32K context; MiniMax H3: local H3-Base FL2VA recipes, assuming an NVMe drive; Nemotron 3.5: Bolt and Super VL on one card with the RAM given; FLUX 3 Image Klein: Black Forest Labs' int8 ComfyUI route on the card. "With caveats" means part of the model sits in system RAM (slower), an unmeasured peak, or a route the checker marks conditional. Speeds appear only where they were measured on this very card in this setup; official estimates and runs on other setups are left out (the full checkers show them, labeled); RigCheck Lab's own runs on this card, each at its own context, are listed below. Pick this card in a checker for your exact setup.

RigCheck Lab measurements

RigCheck measurement RigCheck Lab measured Qwen3.8-27B and Gemma 4 26B-A4B on the RTX 4060 Ti 16 GB itself, with llama.cpp's llama-bench on rented cloud machines with one graphics card each (2026-10-21): all layers on the GPU, flash attention, an f16 KV cache. Each row is one model file at one context on one card, run as separate processes (3 per row); the numbers are the lowest and highest of those runs' averages. They hold only for that file, context, card, host and llama.cpp build: no other setup is shown here.

RigCheck Lab, Qwen3.8-27B and Gemma 4 26B-A4B: tokens per second and peak GPU memory, by model, file and context
ModelFileContextPrompt processing (tok/s)Generation (tok/s)Peak GPU memory (GB)Runs
Qwen3.8-27BUD-IQ3_XXSabout 4K945–94622.911.53 of 3 ran
Qwen3.8-27BUD-IQ3_XXSabout 32K578–57919.813.43 of 3 ran
Gemma 4 26B-A4BUD-IQ3_XXSabout 4K2,907–2,91576.512.63 of 3 ran
Gemma 4 26B-A4BUD-IQ3_XXSabout 32K1,558–1,56265.2–65.313.23 of 3 ran

Context: prompt processing times 512 tokens after 3,584 (about 4K) or 32,256 (about 32K) untimed tokens; generation times 128 tokens after the same untimed tokens. Each run repeats its test 3 times, and those single samples vary more than the averages shown (their standard deviations are in the data). Peak GPU memory is the whole card's memory in use as nvidia-smi reports it (MiB, converted to GB), read about every 200 ms, so short spikes can be missed. Hosts: Host 1, RTX 4060 Ti 16 GB, 2026-10-21: Vast.ai machine 39330, 165 W power limit, PCIe 3.0 ×8, Intel(R) Core(TM) i5-8500 CPU @ 3.00GHz with 6 CPU cores and 33.5 GB of RAM allocated, driver 580.173.02, llama.cpp build 11783 (CUDA). Every run, with its command, file checksum, samples and correction history, is in lab.json (CC BY 4.0); how runs are made and corrected: RigCheck Lab method.

What more RAM would change

Measured a local model on your own hardware, or tried and it failed? Send us the numbers: paste the benchmark output or log into the form (or email it). After checking it we add it to this page, labeled as a community report and shared under CC BY 4.0.

About the RTX 4060 Ti 16 GB

RTX 4060 Ti 16 GB: 16 GB GDDR6 of video memory (NVIDIA GeForce RTX 4060 family specifications). For the large mixture-of-experts models here, system RAM matters as much as VRAM: the parts that do not fit on the card are kept in RAM.

This page is the 16 GB RTX 4060 Ti. The 8 GB version fits less; check it in the home page finder.

FAQ

Can an RTX 4060 Ti 16 GB run Qwen3.8-Flash-Next in the Strata engine?

Yes, with 64 GB of system RAM in RigCheck's quick setup: IQ3_S; No measured speed for this hardware and setup yet; Fits by the docs' rule (experts + ~10 GB), with little room to spare: close the browser while it runs. The full checker takes your exact setup (RAM, disk, operating system, processor) and gives the install steps.

Can an RTX 4060 Ti 16 GB run Mistral Medium 3.5 in llama.cpp?

With caveats, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: UD-IQ3_XXS · GPU and system RAM (16 of 88 layers on the GPU): a dense model runs slowly from RAM; No measured speed for this hardware and setup yet; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM, image input) and gives the command to start it.

Can an RTX 4060 Ti 16 GB run Qwen3.8-27B in llama.cpp?

Yes, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: UD-IQ3_XXS · GPU memory (input embeddings in system RAM); No official or community speed for this setup; RigCheck Lab measured Qwen3.8-27B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM, image input) and gives the command to start it.

Can an RTX 4060 Ti 16 GB run Gemma 4 (E2B, E4B, 12B, 26B-A4B, 31B) in llama.cpp?

Yes, by RigCheck's capacity estimate, with 64 GB of system RAM in RigCheck's quick setup: Gemma 4 31B: fits (capacity estimate) (UD-IQ2_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 26B-A4B: fits (capacity estimate) (UD-IQ3_XXS, GPU memory (input embeddings in system RAM)); Gemma 4 12B: fits (capacity estimate) (Q6_K, GPU memory (input embeddings in system RAM)); Gemma 4 E4B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM)); Gemma 4 E2B: fits (capacity estimate) (BF16, GPU memory (input embeddings in system RAM)); No official or community speed for this setup; RigCheck Lab measured Gemma 4 26B-A4B (UD-IQ3_XXS) on this card at this context: see RigCheck Lab measurements below; RigCheck capacity estimate, not a measured setup. The full checker takes your exact setup (context length, RAM, image input) and gives the command to start it.

Can an RTX 4060 Ti 16 GB run MiniMax H3 (video + audio)?

A published route or recipe matches, with caveats, with 64 GB of system RAM in RigCheck's quick setup: Hardware memory matches a published video recipe; exact clip peak unverified; H3-Base FL2VA, source recipe canvases only. Disk-streaming routes assume an NVMe drive. Host-RAM and lab assumptions remain in the full checker; full 2K needs an API; Matching recipes: SGLang Recipe A (lab): one card, DiT, text encoder and VAE streamed from the checkpoint file on an NVMe drive (Recipe A), BF16; measured at 864×480 (at the native 768P the cookbook says to use video_vae=24 if the decode runs out of memory, not measured); lab run with VRAM capped at 12 GiB and a simulated 32 GB host: shows the memory fits, not your card's speed; a real 32 GB machine re-reads the weights from disk on every step. The full checker lets you change the route or recipe settings and lists the files and the published recipe.

Can an RTX 4060 Ti 16 GB run Nemotron 3.5 (Bolt 74B-A9B, Super VL 120B-A12B)?

A published route or recipe matches, with caveats, with 64 GB of system RAM in RigCheck's quick setup: Nemotron 3.5 Bolt 74B-A9B: a published route or recipe matches, with caveats (Matches Unsloth's short-context budget: about 32 GB of RAM or unified memory); Nemotron 3.5 Super VL 120B-A12B: a published route or recipe matches, with caveats (Matches Unsloth's 80 GB budget: UD-IQ2_M, UD-Q2_K_XL or UD-IQ3_XXS (55.9–59.8 GB)); Nemotron 3.5 Bolt 74B-A9B: NVIDIA publishes no recipe for a 16 GB card; its discrete-GPU recipes start at 32 GB. Unsloth budgets around 32 GB of RAM or unified memory for short-context use (48–64 GB for more headroom) with its 22 GB IQ2_XXS GGUF in llama.cpp; longer contexts, images and video need more. This system RAM is 64 GB. That is Unsloth's planning estimate, not a measured run on this hardware. Nemotron 3.5 Super VL 120B-A12B: Unsloth's planning estimate for a modest context, including the OS, runtime, KV cache and vision file; not a measured run. This setup counts 16 GB of VRAM + 64 GB of RAM = 80 GB. Longer contexts and video need more. The full checker lets you change the route or recipe settings and lists the files and the published recipe.

Can an RTX 4060 Ti 16 GB run FLUX 3 Image Klein (int8, ComfyUI)?

A published route or recipe matches, with caveats, with 64 GB of system RAM in RigCheck's quick setup: Matches BFL's 8 GB guidance: the int8 files in ComfyUI; BFL: the int8 ComfyUI files run on 8 GB GPUs such as the RTX 3070. Image model, int8 text encoder and VAE together are 11.0 GB; by our reading, on a card smaller than that ComfyUI keeps them on the GPU one at a time and needs system RAM for the rest. BFL gives no RAM figure. The prompt upsampler BFL says not to skip is a 9B model (6.72 GB w4a8, 9.75 GB int8 or 18.8 GB BF16 in ComfyUI); BFL does not say whether its 8 GB figure includes it. When it does not fit with the image model, run it on its own first (BFL's README) or switch it off in the ComfyUI template, with weaker results. The full checker lets you change the route or recipe settings and lists the files and the published recipe.

Other graphics cards

Radeon VII / Instinct MI50 16 GB · RTX 3060 12 GB · RTX 4070 Ti · RTX 3090 · RTX 4090 · RTX 5090 · Every 16 GB card · All graphics cards · Any other card or Mac

Sources