01Match a published Kolibri deployment route

Open in a full page ↗

Loading private checker…

Spec sheetSources checked 2026-10-12

Developer

Aleph Alpha Official

Params

78.10B total parameters Official

Active

3.46B active per token Official

Weights

32 downloadable shards total 78.84 GB Official

License

Apache 2.0 Official

Official routes

Aleph Alpha's vLLM plugin and, since October 20, 2026, Transformers 5.20 or newer Official

llama.cpp

requires a custom llama.cpp patch Community

Official minimum

2× A100 80 GB, 2× H100 SXM5, or 1× H200/B200/B300 Official

02Why 3.46B active still needs about 78 GB

The 78,103,074,560 total parameters remain resident or accessible, including experts that are not selected for the current token. Each layer has 384 routed experts and one shared expert; six routed experts are selected per token. Activation count changes computation, not total stored weights. FP8 uses block scales and keeps some tensors at BF16; downloadable size is not exactly one byte per parameter. Official card and configuration

There are 50 layers with four 513-token sliding-window layers per full-attention layer, four KV heads and 128 values per head. Our raw FP8 cache lower bound for one sequence is (10 × context + 40 × min(context, 513)) × 2 × 4 × 128 bytes. It excludes scales, block padding, buffers and runtime choices that may retain more tokens. The community GGUF route uses q8_0 cache, whose block scales differ; this FP8 payload is not its exact allocation. Native context is 262,144; the official card reports extrapolation to 1,048,576 but recommends at most 262,144. The checker does not promise one-million-token serving on minimum hardware.

03Official server and experimental GGUF routes

Official: install aleph-alpha-inference ≥1 or its container, which installs its supported vLLM version. Enable FP8 KV cache, the Kolibri reasoning/tool parsers and auto tool choice as shown in the checker. The launch template sets tensor parallelism to the listed minimum GPU count and caps the requested context and concurrency at one sequence. Extra available cards are not automatically used; check visible devices and the runtime’s allocated cache. These standard flags come from the vLLM parallelism guide and serve arguments, not an additional measured Kolibri configuration. Recommended hardware is 2× H100 SXM5, 2× H200 or 1× B200/B300. Plain stock vLLM is insufficient without the plugin.

Official, Transformers: since the card's October 20, 2026 update, it also says Kolibri “runs with Transformers, which supports it natively from version 5.20” (pip install "transformers[kernels,torch]>=5.20", loading with device_map="auto"). The card's hardware row (the minimum above) is not tied to a runtime, and it gives no separate hardware, memory or speed for Transformers; the checker has no Transformers route. The same FP8 weights must be stored, and Transformers support is not llama.cpp, Ollama or LM Studio support.

Community: apply the provided patch to llama.cpp commit 836d571 and build with your backend, or inspect the author's patched prebuilt release. Use Q4_K_M and the model's embedded chat template. The author tested Ryzen 7 7800X3D with 128 GB DDR5-6400, eight threads, CPU only or RTX 5070 12 GB with attention/shared experts on GPU and routed experts in RAM (--cpu-moe). The long-context GPU runs used q8_0 cache and reached a 6.4 GB VRAM allocation at 262,144 context; this is that setup's result, not a blanket 6.4 GB minimum. The checker explicitly sets eight CPU threads and flash attention; its server templates are not benchmark reproductions. CUDA graph benchmark instability was avoided with GGML_CUDA_DISABLE_GRAPHS=1; the checker offers this as a separate POSIX shell stability variant for the GPU route. The author’s long-context server also succeeded with graphs enabled, so disabling them is not a universal server requirement. Consult the author’s notes and adapt thread counts to your CPU.

Stock llama.cpp does not yet include the architecture in the source checked October 12. Ollama/LM Studio require architecture support in their embedded runtime; a generic Hugging Face “use this model” command is not evidence. Metal and Vulkan were not tested by the converter. Downloading a small quantization alone cannot establish compatibility or original FP8 quality.

04Download choices

Official FP8 files and independent GGUF files, decimal GB (all split parts):

05Community measurements and limitations

Converter's October 7 measurement: Q4_K_M on Ryzen 7 7800X3D, 128 GB DDR5-6400, eight threads, flash attention: CPU-only generation 12.3 ±1.4 tok/s on a 128-token generation benchmark. With RTX 5070 attention/shared experts and routed experts in RAM: 18.0 ±0.3 tok/s; at 32,768 tokens of prior context, 11.7 ±1.1 tok/s. GPU benchmark rows disabled CUDA graphs. No speed is transferred to your selected CPU, another quantization, another GPU or another context. The small reference-logit, smoke and recall tests do not establish broad model quality or safety parity.

Kolibri weights are Apache 2.0; the independent runtime patch is MIT. Follow the official model card's intended-use and limitation guidance. No hardware or cloud referral changes a verdict.

Measured Kolibri-1 on your own hardware, or tried and it failed? Send us the numbers: paste the benchmark output or log into the form (or email it). After checking it we add it to this page, labeled as a community report and shared under CC BY 4.0.

06Common setups

The checker's own answer for 8 common setups (graphics cards with 32 or 64 GB of system RAM, then Macs), in the finder's quick mode (the official FP8 route at a 32K context). Answers are official routes, not measured setups; "Unknown" means the checker has no route or source for that hardware. Your own setup: the checker.

Machine-readable: the same answers as JSON (CC BY 4.0, attribution "RigCheck").

07FAQ

Why does Kolibri need 78 GB if only 3.46B parameters are active?

Active parameters describe which experts compute a token. All 78.10B total weights must still be stored or streamed. Official FP8 weight memory is about 78 GB, with additional context and runtime overhead.

Can Kolibri run on a gaming GPU?

The official minimum uses server accelerators. A community patched GGUF recipe uses RTX 5070 12 GB and 128 GB system RAM, with routed experts on CPU. Its speed depends on that CPU and RAM bandwidth; other cards do not inherit a verified result.

Can I use Kolibri GGUF in Ollama or on a Mac?

Not based on file capacity alone. The converter requires a custom llama.cpp architecture patch; stock Ollama integration and Metal execution are not verified here. The author did not test Metal or Vulkan.

Sources

• Official card, config, license and file list; official inference plugin official
• Independent conversion, patch, files and measured runs community
• Stock architecture registry; H100 SXM specifications (80 GB)
• All sources checked October 12, 2026. Cache arithmetic is a RigCheck raw-payload lower bound, not a serving-memory estimate.