Lecture 10 - DINO-WM from Scratch | Planning in Feature Space

In this lecture, we build DINO-WM (Paper: https://arxiv.org/abs/2411.04983): a frozen DINOv2 encoder, a transformer predictor with a block-causal attention mask over 3 × 264 tokens, and model-predictive control (CEM) to plan from a start frame to a goal frame. We train it on the Push-T task, roll it out autoregressively, and plan entirely in imagination. Project: Build DINO-WM from scratch, then train it on real Push-T robot data and plan a trajectory zero-shot in imagination https://github.com/RajatDandekar/build-a-world-model-from-scratch/tree/main/lecture-10-dino-wm All slides, code &