Neel Nanda

Note: Being badly named doesn't mean current "unlearning" work is useless, I just think it should be renamed to eg "knowledge suppression" and people should be realistic about its limitations and how robust it's likely to be

This looks like a lovely paper! I've long thought that the term "unlearning" was BS. IMO unlearning means removing knowledge from the model's weights. But people tend to measure behaviour, which is indistinguishable from suppression in later layers. I'm glad someone showed this!

I really hope OpenAI can work these issues out, with or without Sam. AGI is a profound and incredibly consequential technology, and I want all companies working on it to be as sensible and well-functioning as possible.

Like, regardless of whether you think x-risk is a bunch of sci-fi nonsense, that is just not a well run company, and it is the job of a board to do things about that.