The world's smallest transformer model

This is the second of a trilogy of videos titled "The world's smallest transformer model". In this video, we build a tiny transformer model with one hidden layer of 2 nodes, and no bias or activation function. This transformer is able to form small sentences based on a dataset of two sentences. Video 1: How neural networks bend space https://www.youtube.com/watch?v=3THzlSIzfug Video 2: The world's smallest transformer model (this one) Video 3: The world's smallest attention mechanism https://www.youtube.com/watch?v=J4hSc9He614 ADDENDUM: Key and Query Matrices worked out in detail https://ww