They Shrunk Qwen 27B to 7.9GB... Here's What It Lost

Underdog Saluki 27B is a new 7.9GB local AI model that compresses the original 55.6GB Qwen 27B model using aggressive 2-bit quantization, yet reportedly outperforms the full-size model at tool calling. But there's a catch. In this video, I test Saluki 27B locally using llama.cpp, parallel tool calls and JSON output, memory usage and inference performance, and show how such a heavily compressed LLM can achieve higher tool-calling benchmark scores while losing accuracy in math, coding, and reasoning. 🔗 Relevant Links Hugging Face - https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0 ❤️