RLMs are Compositional Generalizers with Alex Zhang from MIT

we're back with Alex L. Zhang, MIT CSAIL 2nd year PhD student and creator of Recursive Language Models, to talk about compositional generalization and locally in-distribution. as a refresher RLMs put the prompt in a python REPL instead of feeding it to the model, letting it peek, grep, chunk and recursively call itself on the pieces. the new update is that an RLM trained on short tasks only can generalizes to tasks 8-32x longer and even to new domains, while the vanilla transformer stays can't. we get into what "locally in-distribution" really means, alex's mismanaged geniuses hypothesis