Low Rank Adaptation (LoRA) - How to search for a needle in a much smaller haystack

When you want to fine-tune a model with many parameters and you want to do it fast and effectively, Low Rank Adaptation (LoRA) is there to help you. With LoRA, you trust that the base model is in a good state, and so you assume there may already be many dependencies among the parameters. Thus, you look for better parameters in a much smaller dimensional space. Learn all about LoRA in this friendly video! CORRECTION: (thanks @ryanhewitt9902!) At 30:14 the vector on the right should be [pa, qa, ra, sa, ta], instead of [p/a, q/a, r/a, s/a, t/a].