colibri

What is colibri?

colibri is a tiny inference engine, written in pure C with no dependencies, that runs very large open mixture-of-experts language models on ordinary hardware. It keeps the part of the model used for each token in memory and streams the rest, the experts, from disk when needed, so models with hundreds of billions of parameters can run without a GPU. It supports several open model families, including GLM, Kimi, DeepSeek, Qwen and MiMo, plus an image model. You need at least 8 GB of RAM, and more is better. It is free and open source under the Apache 2.0 licence.