QuixiAI Integrates GLM-5.2-Vision Model Into Llama.cpp For 100 Tokens Per Second Decoding

aiai-infrastructureai-inference-platformsai-modelingai-open-models 4 posts · 2 accounts

QuixiAI integrates support for the GLM-5.2-Vision language model into the llama.cpp open-source inference library and publishes the associated GGUF weight files. The release, submitted via a pull request, enables local deployment of the vision-capable system with a dedicated toggle to switch between text and vision processing.

The updated build supports the exl3 format and vLLM, according to technical details shared alongside the model. The configuration achieves a decoding throughput of 100 tokens per second on four RTX 6000 GPUs. The project acknowledges foundational optimization efforts by Baseten and developer Brandonmusic.

From the sources (4 posts)

@0xsero

GLM-5.2 - vision enabled, on/off switch, exl3, vllm. 100 tok/s decode + 0.104 KLD built on Brandonmusic's and Baseten's work. Full reproducible recipe in the model card. 4x 6000s & 3x Sparks enjoy. https://t

@quixiai

GLM-5.2-Vision is in to llama.cpp @ivanfioravanti game on!

@quixiai

Thank you @PaIbraNiang1 and @baseten for your pioneering work! I've published To bring it together to llama.cpp

@quixiai

I was trying to upload QuixiAI/GLM-5.2-Vision-GGUF and I was limited by @huggingface api. @ClementDelangue this is an important model. the llama.cpp PR is here:

Anthropic Includes Claude Fable 5 in Max and Team Premium Plans Starting July 20 at 50% of Limits Poolin Files Chapter 11 Bankruptcy And Plans $52 Million Sale Of Texas Bitcoin Mining Sites