Language models recognize and inhibit interventions to their activations

First, we show that language models can detect, localize, and verbalize the difference between perturbations applied to their activations. Second, they can even (be interpreted as) control(ling) them. More precisely, we perturb the activations of a language model by applying dropout, adding Gaussian noise, or adding steering vectors, and ask it, e.g., "Which perturbation was applied?". All the models we test can easily detect and localize the perturbations. Notably, the accuracy of two models in identifying which perturbation is applied correlates with the perturbation strength. Most models ca