This question inspired LoRA fine-tuning

Fireworks is offering $100 in free credits for 1,000 of my Patreon members (including free tier). Claim your free Fireworks credits here: https://forms.gle/vmQ5Xhv4EaxHuhLJA Why does LoRA work with rank 1? Fine-tuning, intrinsic dimension, and the geometry of low-rank adaptation LoRA (Low-Rank Adaptation) is the most popular way to fine-tune LLMs — it can teach a 2.8-trillion-parameter model like Kimi K3 a new skill by training less than 0.2% of its weights. But why does such a tiny rank work? This video builds the geometric intuition from scratch: what full fine-tuning actually does to the lo