Sydney Katz: What Is Mechanistic Interpretability?

Sydney Katz, Lecturer in Aeronautics & Astronautics at Stanford University, explains mechanistic interpretability, a fast-growing area of AI research that opens up the “black box” of neural networks. Katz says that by looking inside a model, researchers can find the patterns driving its decisions. She uses a runway example: a camera-based system that helps guide an aircraft must focus on the right features, not a bird or a shadow on the runway. Interpretability techniques let engineers check this and build justified confidence in safety-critical AI. Validation of Safety Critical Systems (AA22