I run 177B model on SSD (It's Fast!) | Qwen 3.8 Flash Next Test
Can you run a massive 177B model on SSD? In this video, I'm running Qwen 3.8 Flash Next IQ4XS 85GB disk size on my AI rig (4060 8GB + RTX 5060 Ti 16GB + 32GB RAM) We'll see if offloading the embedded n-gram table to an NVMe SSD can still run at usable speed. Follow me on X: https://x.com/redStapler_twit *** My Portable AI PC Setup *** CPU: https://amzn.to/3Lx52Vv GPU: https://amzn.to/3YWMPDS RAM: https://amzn.to/49l7vdD Board: https://amzn.to/4poMEfp PSU: https://amzn.to/4po6xU0 ⏱️ TIMESTAMPS 00:00 - Intro 00:28 - Qwen 3.8 Flash-Next Architecture 01:06 - N-Gram & SSD Offloading 01:38 - Llama.c
Discussion
Rejoignez la communauté pour réagir et poser vos questions.
Aucun message pour l'instant. Lancez la discussion !