{"rewrite":{"id":"r_7e69fa9e846d712ddfee9c60","clusterId":"c_bde5dfd4963c3709df4d4396","slug":"colibri-inference-engine-runs-744b-parameter-glm-5-2-on-a-consumer-pc-with-25gb-of-memory","model":"deepseek-v4-flash","headline":"Colibrì Inference Engine Runs 744B-Parameter GLM-5.2 on a Consumer PC with 25GB of Memory","summary":"A new inference engine called Colibrì can run the 744-billion-parameter GLM-5.2 model on a consumer PC with 25GB of RAM by streaming expert data from an SSD. The engine exploits the model's Mixture-of-Experts architecture, activating only about 40 billion parameters per token. Speed is very slow-0.05 to 0.1 tokens per second-but the approach makes large open models practical on limited hardware.","whyItMatters":"Colibrì demonstrates that even a 744B-parameter open model can be run on a consumer PC by trading speed for memory, potentially broadening access to large language models beyond users with expensive hardware.","webCardHtml":"\u003cp\u003eThe developer vforno built Colibrì around GLM-5.2\u0026#39;s Mixture-of-Experts structure, which activates only a subset of parameters per token. The engine keeps about 9.9GB of always-needed data in RAM in 4-bit format, while 21,504 experts totaling 370GB reside on an NVMe SSD. An LRU cache and a learning-based cache reduce SSD reads by keeping frequently used experts in RAM. The engine is a single C file with no external dependencies and does not require a GPU. The trade-off is speed: on a 12-core CPU with 25GB RAM, generation runs at 0.05-0.1 tokens per second when the cache is empty, meaning a short response can take several minutes.\u003c/p\u003e","blueskyPost":"Colibrì runs the 744B-parameter GLM-5.2 on a consumer PC with 25GB RAM by streaming experts from SSD. Speed is very slow, but it makes huge open models usable on ordinary hardware.","twitterPost":"Colibrì runs the 744B-parameter GLM-5.2 on a consumer PC with 25GB RAM by streaming experts from SSD. Speed is very slow, but it makes huge open models usable on ordinary hardware.","threadsPost":null,"newsletterBlurb":"A new inference engine called Colibrì can run the 744-billion-parameter GLM-5.2 model on a consumer PC with 25GB of RAM by streaming expert data from an SSD. The engine exploits the model's Mixture-of-Experts architecture, activating only about 40 billion parameters per token. Speed is very slow-0.05 to 0.1 tokens per second-but the approach makes large open models practical on limited hardware.","attributionJson":"[{\"source\":\"GIGAZINE\",\"url\":\"https://gigazine.net/news/20260710-colibri-glm/\",\"title\":\"Inference Engine \\\"Colibrì\\\" Emerges to Run 744-Billion-Parameter Giant AI \\\"GLM-5.2\\\" on an Ordinary PC with 25GB of Memory\"}]","lintFlagsJson":null,"lintHits":0,"costUsd":0,"inputTokens":4314,"outputTokens":1430,"status":"published","repairAttempts":0,"nextRepairAt":null,"factsAttemptedAt":1783757008,"createdAt":"2026-07-11T07:52:29.000Z","publishedAt":"2026-07-11T07:57:28.000Z","updatedAt":"2026-07-11T07:52:29.000Z"},"cluster":{"id":"c_bde5dfd4963c3709df4d4396","canonicalTitle":"7440億パラメーターの巨大AI「GLM-5.2」をメモリ25GBの普通のPCで動かす推論エンジン「Colibrì」が登場","representativeArticleId":"a_27e37d311d7465a376f96ef7","sourceCount":1,"writtenSourceCount":1,"writeAttempts":0,"isSolo":true,"entitiesJson":"{\"anime_titles\":[],\"manga_titles\":[],\"work_titles\":[],\"studios\":[],\"people\":[],\"type\":\"news\",\"domain\":\"other\",\"is_roundup\":false}","contentType":"news","status":"published","firstSeenAt":"2026-07-10T07:20:00.000Z","lastSeenAt":"2026-07-10T07:20:00.000Z","updatedAt":"2026-07-11T07:57:29.000Z"},"attribution":[{"source":"GIGAZINE","url":"https://gigazine.net/news/20260710-colibri-glm/","title":"7440億パラメーターの巨大AI「GLM-5.2」をメモリ25GBの普通のPCで動かす推論エンジン「Colibrì」が登場"}],"entities":{"anime_titles":[],"manga_titles":[],"work_titles":[],"studios":[],"people":[],"type":"news","domain":"other","is_roundup":false},"keyFacts":null}