AgentFlare

Scholar · mechanistic interpretability neural networks

Mechanistic Interpretability: Research Digest

10 foundational papers with 3,902 combined citations, plus the latest arXiv preprints.

Foundational papers103,902 citations
Most citedProgress measures for grokking via me…1,214 citations · 2023
arXiv, Sep 202640 papers+8% vs Oct 2025
Newest preprint2026-10-15

AISA Scholar API · arXiv

arXiv submissions per monthPapers matching abs:"mechanistic interpretability" · last 12 complete months
monthly arXiv submissions: Oct 2025 37 → Sep 2026 40; high 77 (May 2026), low 23 (Dec 2025)19406081Oct 2025Mar 2026Sep 2026
402 · Pay-per-Crawl$0.02

You're reading the full report — free for humans and search engines. The same content ships as a structured JSON dataset (normalized facts, sources and citation arrays, no HTML parsing) so AI agents can consume it directly — metered per fetch via the open HTTP 402 payment protocol.

⚡ Agents: fetch .data.json · $0.02 → agents: crawlerAutoPrice=true · verify at https://esa.aisa.one/api/v1/access/verify

What’s new on arXiv

A12026-10-15cs.CV

Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis

Joshua Muhumuza, Cornelius Kalenzi

Vision Transformers deployed in medical imaging exploit spurious correlations, yet the internal mechanisms remain unidentified and impossible to fix. We apply mechanistic interpretability to locate one such mechanism in ViT-Base 16: a single attention head (L8H10) that functions as a dedicated annotation-reader on chest X-rays, attending to corner text rather than lung pathology in 62% of 200 COVID-19 images. Causal validation via inpainting confirms annotation content rather than luminance or edge density drives the behaviour, increasing lung attention 1.52 times upon removal in affected images (p<0.0001). The finding replicates across NIH ChestX-ray14 (79%), DeiT-Base (88%), and ViT-Large 16 (80%), establishing spurious annotation-attending heads as a consistent property of ViT-family architectures across two datasets and three architectures. Full fine-tuning does not suppress the behaviour and may entrench it: 98.4% of spurious images remain spurious and the spurious rate rises marginally, as the shortcut is reinforced rather than removed. Critically, single-head ablation produces p=0.340 and multi-head ablation of all three identified carriers produces p=0.532, meaning a causally verified annotation-reading circuit is completely invisible to output-level evaluation. Mechanistic auditing finds what behavioural testing cannot.

A22026-10-15cs.LG

Learning and Forecasting Multiscale Spatiotemporal Epidemic Dynamics with Lagrangian-Eulerian Graph Neural ODEs

Jinyi Duan, Qian Li, Biao Tang

Accurate medium- to long-range epidemic forecasting requires extrapolating high-dimensional, heterogeneous and nonstationary dynamics from partial observations. Yet hybrid models often leave implicit which observations inform which epidemic processes and how learned graph interactions modify system state. We introduce LE-EpiGNODE, which replaces generic message passing with process-specific, mechanism-guided neural operators: age-contact information drives Lagrangian infection generation, whereas mobility drives conservative Eulerian population transport within a structure-preserving universal differential equation. We evaluated LE-EpiGNODE across 14 controlled synthetic mechanisms spanning time-modulated, state-dependent, nonlinear and coupled dynamics, three complementary COVID-19 settings, 21 comparator models and five targeted ablations. Forecasting gains tended to become more pronounced at longer horizons; at the longest horizon, LE-EpiGNODE achieved the lowest WMAPE in all three real-world datasets, reducing error by 17.7-40.0% relative to the best baseline. Controlled ablations linked these gains to the Lagrangian-Eulerian decomposition and process-aligned priors, while synthetic ground truth showed that learned operators and induced state fluxes track known generating dynamics. These results support mechanism-organized graph neural dynamics as a route to reliable long-horizon extrapolation with directly testable, process-level mechanistic interpretation beyond trajectory accuracy.

A32026-10-14stat.ML

Complete or Sparse: A Tale of Two Identifiabilities

Junze Zhou, David Klindt

Understanding when neural networks learn interpretable representations is central to mechanistic interpretability. We study this through identifiability: when can supervised learning recover the true latent variables underlying the data? We introduce two data generating processes distinguished by their causal direction where either labels cause latents (generative) or latents cause labels (discriminative). When the representation has at least as many dimensions as the latent space, we prove that the optimal encoders are linear in the latents. Conversely, when dimensions are insufficient, identifiability generically fails and encoders become nonlinear. We then show that additional constraints yield recovery of individual latent variables, not just linear mixtures. In the generative setting, non-negative activations and energy efficiency force each neuron to encode exactly one latent, providing a precise account of selective grandmother cell-like responses. In the discriminative setting, when latents are sparse and the true measurement matrix satisfies the spark condition, the learned encoder inherits compressed sensing structure, enabling sparse dictionary learning to perfectly recover individual latents. This constitutes, to our knowledge, the first identifiability result for undercomplete nonlinear independent component analysis. Our framework unifies nonlinear ICA, compressed sensing, and neural network interpretability, delineating exactly when representations transition from nonlinear distortions to linear representations and, under additional constraints or steps, to clean, axis-aligned codes. Theoretical predictions are validated on synthetic benchmarks and on activations drawn from vision models, large language models, and macaque inferotemporal cortex.

A42026-10-14cs.LG

Interpreting Language Model Parameters

Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin, Bart Bussmann, Nathan Hu, Michael Ivanitskiy, Linda Linsefors, Lee Sharkey

Mechanistic interpretability aims to uncover how neural networks use their parameters to implement their impressive neural algorithms. Although previous work has uncovered substantial structure in the intermediate representations that networks use, little progress has been made to understand how the parameters and nonlinearities of networks perform computations on those representations. In this work, we present a method that brings us closer to this understanding by decomposing a language model's parameters into subcomponents that each implement only a small part of the model's learned algorithm, while simultaneously requiring only a small fraction of those subcomponents to account for the network's behavior on any input. The method, adVersarial Parameter Decomposition (VPD), optimizes for decompositions of neural network parameters into simple subcomponents that preserve the network's input-output behavior even when many subcomponents are ablated, including under ablations that are adversarially selected to destroy behavior. This encourages learning subcomponents that provide short, mechanistically faithful descriptions of the network's behavior that should aggregate appropriately into more global descriptions of the network's learned algorithm. We study how sequences of interactions between these parameter subcomponents produce the network's output on particular inputs, enabling a new kind of 'circuit' analysis. While more work remains to be done to deepen our understanding of how neural networks use their parameters to compute their behavior, our work suggests an approach to identify a small set of simple, mechanistically faithful subcomponents on which further mechanistic analysis can be based.

A52026-10-14cs.CL

Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility

Salima Lamsiyah, Adelaide Danilov, Oleksandr Marchenko Breneur, Aria Nourbakhsh

Sparse autoencoders (SAEs) have emerged as a promising approach to mechanistic interpretability by decomposing language-model activations into sparse, potentially interpretable features. However, differences in design assumptions, evaluation objectives, and experimental settings complicate conclusions about their effectiveness. We present a critical survey of 138 studies on SAEs in natural language processing. Our contributions are fourfold: (1) we propose a task-grounded taxonomy connecting dictionary design, feature operations, and NLP functions; (2) we examine how sparsity allocation, geometry, data coverage, and inference affect feature recoverability; (3) we assess evidence for semantic interpretability, mechanistic faithfulness, and behavioral utility; and (4) we synthesize findings across multilingual processing, retrieval, reasoning, classification, and behavioral regulation, highlighting the roles of objectives, baselines, supervision, and intervention pipelines. Through comparative analyses of published results, we reconcile apparent disagreements, identify unresolved questions, and propose directions for more rigorous evaluation. An anonymous research register https://anonymous.4open.science/r/sae-research-register-C331/register.html accompanies the survey.

A62026-10-14cs.AI

Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability

Anton Korznikov, Alexey Dontsov, Elena Tutubalina

Mechanistic interpretability aims to decompose neural network computations into human-understandable features, yet progress is limited by the absence of ground-truth units to evaluate against. We address this with a Sparse Transformer (ST): a 5-layer language model trained on TinyStories whose wide, TopK-sparse MLP forces neuron specialization. We show that ST neurons are highly interpretable: they outperform a loss-matched dense transformer by 43% on AutoInterp, and circuits of a median of 12 neurons suffice to reproduce individual model predictions. This testbed allows us to directly compare feature learning methods: Transcoder, which approximates the MLP input-output mapping, recovers ST's neuron vectors with 52.6% higher cosine similarity than Sparse Autoencoder, which only reconstructs outputs. Furthermore, the ST enables a controlled study of polysemanticity: distilling its sparse MLP into a dense counterpart reveals how compression leads to feature superposition. These results show how a known reference basis sharpens evaluation and understanding, a role we hope the ST can serve for the field.

A72026-10-13cs.AI

Monitoring latent reasoning in looped language models

Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk, Michal Byra, Neo Christopher Chung

Looped language models (LMs) reuse one block of layers several times per token generation, promising more latent computation at no additional storage cost. Despite their rising popularity, the analysis of looped LMs is still in its infancy: little is known about whether the iterative transformations actually improve their capabilities, or whether these models remain stable under prompt perturbation. We construct an inspection protocol rooted in Mechanistic Interpretability comprising five diverse lenses. We apply it to examine eight looped LMs from four families. Models differ in many aspects but the key one is whether the prompt is transmitted only within the looped state (carry loops: Ouro, LoopUS) or re-supplied at every iteration (re-read loops: Parcae, and Rec-OLMo & Huginn from the Raven family). The results of our study provide valuable insights concerning safety and efficient use of looped language models.

A82026-10-09cs.AI

Neural networks are more modular than single neurons suggest

Shreya Saha, Benjamin Bergen, Meenakshi Khosla

Neural networks routinely develop representations in which individual neurons contribute to multiple distinct functions, leading to the widespread view that such networks lack modular organization. We challenge this conclusion by introducing a formally nested hierarchy of modularity: unit-level, which requires distinct computations to occupy disjoint neurons; orthogonal, which asks whether a rotation reveals functionally independent subspaces; and linear, which asks whether a bounded linear map can separate task-relevant computations into independent subspaces. We develop a unified framework for learning decompositions at each level and evaluating them causally through selective ablation. Across vision models, language models and multi-task Recurrent Neural Networks (RNNs), we find that networks which lack unit-level modularity nonetheless often exhibit clear functional modularity at the subspace level. In vision and language models, content versus style and syntax versus semantics respectively dissociate under subspace ablation despite sharing overlapping populations of neurons. These learned subspaces also align differentially with brain data, with content and style subspaces predicting higher and lower visual cortex respectively, and semantic subspaces outperforming syntactic ones in predicting language cortex activity. In RNNs trained on multiple cognitive tasks and previously shown to lack modular structure, our decomposition recovers sharp double dissociations between tasks. The absence of unit-level modularity frequently coexists with clear subspace-level modularity across domains, enabling targeted intervention on specific computations even in networks where no individual neuron is specialized, with broad implications for mechanistic interpretability and for understanding functional organization in both biological and artificial neural systems.

Foundational papers

  1. 2023
    Towards automated circuit discovery for mechanistic interpretability996 citations
  2. 2023
    Progress measures for grokking via mechanistic interpretability1,214 citations
  3. 2024
    Mechanistic interpretability for AI safety--a review711 citations
  4. 2024
    A practical review of mechanistic interpretability for transformer-based language models238 citations
  5. 2025
    Open problems in mechanistic interpretability320 citations
  6. 2025
    Everything, everywhere, all at once: Is mechanistic interpretability identifiable?43 citations
  7. 2025
    Causal abstraction: A theoretical foundation for mechanistic interpretability279 citations
  8. 2026-10
    Latest on arXiv: Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation…cs.CV
Citations per paperCitation counts via the AISA Scholar API
citations: R1 Mechanistic interpret… 711, R2 Towards automated cir… 996, R3 Open problems in mech… 320, R4 Bridging the black bo… 46, R5 Exploring mechanistic… 50, R6 Progress measures for… 1,214, R7 Everything, everywher… 43, R8 A mathematical philos… 5, R9 Causal abstraction: A… 279, R10 A practical review of… 238R1 Mechanistic interpret…711R2 Towards automated cir…996R3 Open problems in mech…320R4 Bridging the black bo…46R5 Exploring mechanistic…50R6 Progress measures for…1,214R7 Everything, everywher…43R8 A mathematical philos…5R9 Causal abstraction: A…279R10 A practical review of…238
R12024711 citations

Mechanistic interpretability for AI safety--a review

L Bereska, E Gavves

… This review explores mechanistic interpretability: reverse engineering the computational mechanisms and representations learned by neural networks into human-understandable …

R32025320 citations

Open problems in mechanistic interpretability

L Sharkey, B Chughtai, J Batson, J Lindsey et al.

… , our aim is to decompose a neural network and study its parts in isolation in order to explain … mechanistic interpretability work; we primarily focus on understanding neural network …

R620231,214 citations

Progress measures for grokking via mechanistic interpretability

N Nanda, L Chan, T Lieberum, J Smith et al.

… Neural networks often exhibit emergent behavior, where qualitatively new capabilities arise … We argue that progress measures can be found via mechanistic interpretability: reverse…

How this page is made. Foundational papers and citation counts come from academic search via the AISA Scholar API, with abstracts read from each paper’s page; recent preprints and monthly volumes come from the public arXiv API (metadata CC0). The digest is written by a language model from those abstracts only — every theme links the papers it draws on, and figures not in the inputs are removed.

Sources & citations

  1. Mechanistic interpretability for AI safety--a review 711 citations
  2. Towards automated circuit discovery for mechanistic interpretability 996 citations
  3. Open problems in mechanistic interpretability 320 citations
  4. Bridging the black box: a survey on mechanistic interpretability in AI 46 citations
  5. Exploring mechanistic interpretability in large language models: Challenges, approaches, and insights 50 citations
  6. Progress measures for grokking via mechanistic interpretability 1,214 citations
  7. Everything, everywhere, all at once: Is mechanistic interpretability identifiable? 43 citations
  8. A mathematical philosophy of explanations in mechanistic interpretability 5 citations
  9. Causal abstraction: A theoretical foundation for mechanistic interpretability 279 citations
  10. A practical review of mechanistic interpretability for transformer-based language models 238 citations
  11. Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis arXiv · 2026-10-15
  12. Learning and Forecasting Multiscale Spatiotemporal Epidemic Dynamics with Lagrangian-Eulerian Graph Neural ODEs arXiv · 2026-10-15
  13. Complete or Sparse: A Tale of Two Identifiabilities arXiv · 2026-10-14
  14. Interpreting Language Model Parameters arXiv · 2026-10-14
  15. Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility arXiv · 2026-10-14
  16. Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability arXiv · 2026-10-14
  17. Monitoring latent reasoning in looped language models arXiv · 2026-10-13
  18. Neural networks are more modular than single neurons suggest arXiv · 2026-10-09

Related