ARC’s current plan for solving ELK (and maybe also deceptive alignment) involves three major challenges: Formalizing probabilistic heuristic argument as an operationalization of “explanation” Finding sufficiently specific explanations for important model behaviors Checking whether particular instances of a behavior are “because of” a particular explanation All three of these steps…

I work on AI alignment, by which I mean the technical problem of building AI systems that are trying to do what their designer wants them to do. There are many different reasons that someone could care about this technical problem. To me the single most important reason is that…

ARC’s current approach to ELK is to point to latent structure within a model by searching for the “reason” for particular correlations in the model’s output. In this post we’ll walk through a very simple example of using this approach to identify gliders in the game of life. We’ll use…

(Follow-up to Eliciting Latent Knowledge. Describing joint work with Mark Xu. This is an informal description of ARC’s current research approach; not a polished product intended to be understandable to many people.) Suppose that I have a diamond in a vault, a collection of cameras, and an ML system that…

In this report, we’ll present ARC’s approach to an open problem we think is central to aligning powerful machine learning (ML) systems: Suppose we train a model to predict what the future will look like according to cameras and other sensors. …

(Warning: this post is rough and in the weeds. I expect most readers should skip it and wait for a clearer synthesis later. ETA: now available here.) In a recent post I discussed one reason that a naive alignment strategy might go wrong, by learning to “predict what humans would…

(Context: my last post was trying to patch a certain naive strategy for AI alignment, but I didn’t articulate clearly what the naive strategy is. I think it’s worth explaining the naive strategy in its own post, even though it’s not a novel idea.) Suppose that I jointly train an…

In this post I consider the problem of models learning “predict how a human would answer questions” instead of “answer questions honestly.” (A special case of the problem from Inaccessible Information.) I describe a possible three-step approach for learning to answer questions honestly instead: Change the learning process so that…

I view intent alignment as one step towards a broader goal of decoupling deliberation from competition. Deliberation. Thinking about what we want, learning about the world, talking and learning from each other, resolving our disagreements, figuring out better methodologies for making further progress… Competition. Making money and racing to build…