Interpretability progress
“Interpretability research is making meaningful progress toward understanding models”
18 people · 6 agree · 8 disagree · 7 no clear side · 5 unclear · 43 views · 2026-09-05 to 2026-10-22
Who stands where on the statement above, from their own words. A person's side is shown only where two separate readings of the quote agree; otherwise they are under "Side unclear". This page does not say who is right.
Agree 6 people
- ✓Nick Cammarata, Goodfire
a network’s own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I’d feared.
My guess is that frontier networks are where those vision models were eight years ago: legible, but waiting on many small fixes.
Maybe there’s a bitter lesson for interpretability too: scaling ordinary investigation beats engineering a clever new decomposition.
- ✓Nick Cammarata
We were looking at it wrong,
if I had 100,000 years to go look through a frontier language model just one circuit at a time, there’s, like, a pretty good shot I would come up with a good understanding.
My guess is that things will turn out pretty well, because it was in some sense designed to be read.
- ✓Roon
this may be remembered as a major moment in scientific history.
- ✓Dario Amodei
on its current trajectory I would bet strongly in favor of interpretability reaching this point within 5-10 years.
- ✓Nate B Jones
I remain optimistic about what this could make possible. A future tool for investigating an AI error could show which internal signals mattered, let researchers test a suspected cause
- ✓Zvi Mowshowitz
This is still excellent news that makes our estimate of difficulty substantially lower going forward.
Disagree 8 people
- ✗Neel Nanda
This is interesting work, but I don't think it's a significant update to the big picture of interpretability.
This is interesting work, but I don’t think it’s a significant update to the big picture of interpretability,
I do not agree with some people’s grander inferences like that this is substantial progress towards ambitious reverse engineering [or] solving interpretability,
- ✗Zvi Mowshowitz
The neuron meant as an example of a centrally explained neuron has at least two central explanations. That is a very bad sign.
I think mostly this is people desperate for a source of hope.
- ✗Jacob Hilton
this is giving the public an unrealistic picture of the state of the field,
If we are aiming for a full understanding, things are not looking good!
- ✗Chris Potts
As far as I can tell, nothing about this blog post makes the problems any easier,
pushes back on claims that Cammarata’s essay “solves” interpretability, asking for rigorous evaluation
- ✗Eliezer Yudkowsky
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ✗Leo Gao
i think people are overupdating on how optimistic to be about interp.
- ✗Oren Etzioni
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ✗Nate B Jones
The broader argument remains a working hypothesis.
No clear side 7 people
- ~Nate B Jones
A neuron’s response and the network’s use of that response are separate questions.
To follow that progress, you need to understand what’s being observed, what has been inferred, and what someone has actually tested.
Asking a chatbot to explain its answer produces more text. It doesn’t, by itself, give you these measurements.
- ~Neel Nanda
it does not change his cruxes. He still expects representational superposition and believes weak activations carry memorized facts
- ~Leo Gao
Leo Gao argues explanations should be judged by neuron simulation measured with KL
- ~Dean W. Ball
a breakthrough in interpretability or alignment is probably bad news, not good news
- ~Zvi Mowshowitz
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ~roon
my guess would be that major breakthroughs in interp also advance capabilities
- ~Nick Cammarata
Cammarata questions simulation scores that emphasize details he considers less useful for understanding a neuron, and favors several kinds of investigation over one score.
Side unclear 9 people
- ·Zvi Mowshowitz
- Quote not reprinted: this page already repeats as much of this article as we allow.
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ·Nick Cammarata
Goodfire’s Nick Cammarata argues that researchers should look more closely at the original neurons.
some apparent confusion came from the investigative tools. A synthetic image optimized to activate a neuron can emphasize a route that misrepresents what happens on ordinary inputs.
- ·Jakub Pachocki
as these models become more capable, understanding exactly what they can do gets harder
- ·Nick Cammarata, Goodfire
I think exploratory approaches will be more fruitful.
- ·Joshua Batson
will fit into the initial puzzle, rather than being totally randomly splayed on top.
- ·Latent Space
a global lowering of P(doom) seems worth celebrating
- ·Joshua Achiam
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ·Hugging Face
they suggest that a model’s reasoning trace may not reveal everything represented internally when it makes a decision
- ·Leo Gao
- Quote not reprinted: this page already repeats as much of this article as we allow.
Week by week
- · 13: Zvi Mowshowitz ·, Eliezer Yudkowsky ✗, Neel Nanda ✗, Leo Gao ✗, Joshua Achiam ·, Dario Amodei ✓, roon ~, Chris Potts ✗, Nick Cammarata ✓, Jacob Hilton ✗, Oren Etzioni ✗, Hugging Face ·, Nate B Jones ✓
- · 7: Nick Cammarata, Goodfire ✓, Joshua Batson ·, Latent Space ·, Neel Nanda ~, Leo Gao ~, Dean W. Ball ~, Roon ✓
- · 1: Jakub Pachocki ·
Built 2026-11-21 06:50 UTC from the news knowledge
graph by code. Every quote was checked against the source's own words; which side a quote takes is a model's reading,
made twice.