AgentFlareScholar · AI alignment and safety
AI Alignment & Safety: Research Digest
10 foundational papers with 916 combined citations, plus the latest arXiv preprints.
AISA Scholar API · arXiv
You're reading the full report — free for humans and search engines.
The same content ships as a structured JSON dataset (normalized facts, sources and
citation arrays, no HTML parsing) so AI agents can consume it directly — metered per fetch via the open
HTTP 402 payment protocol.
crawlerAutoPrice=true · verify at https://esa.aisa.one/api/v1/access/verify
What’s new on arXiv
Evaluating Model Capabilities After Benchmark Saturation
Morio Hamada, Jehyeok Yeon, Matteo Migliarini, Justin Olive, Magda Dubois
Rapid advances in AI capabilities are causing benchmarks to saturate. Once several models approach perfect scores, a benchmark loses its power to differentiate between them, and with it its utility for informing AI safety cases and frontier policies. Yet benchmark items produce far more information than the scores we typically extract from them: a full transcript of how each model solved the task. We investigate whether these transcripts can be used to assess model capabilities even after benchmark scores have saturated. We simulate saturation retrospectively using historical benchmark and model releases, and test whether protocols that access only correctly solved transcripts from saturated benchmarks can predict capabilities measured on held-out, unsaturated items. We evaluate representative contemporary approaches for extracting information from model transcripts: pairwise judging, single-transcript absolute scoring, and predictions from deterministic transcript features or embeddings. We find that saturated transcripts retain capability information beyond benchmark accuracy. Amongst our evaluated protocols, pairwise judging performs best, reducing prediction error by 5.3 times compared to a baseline using only saturated benchmark scores. This capability-relevant information is distributed across transcript sections, but concentrated in specific channels of information such as reasoning and tool-calls. However, the predictive power of protocols does not always generalise across distribution shifts -- for example by underestimating the capabilities of the first reasoning models introduced in early 2025. Saturated benchmarks thus retain capability-predicting signal in their task transcripts, but extending the lifetime of existing benchmarks requires methods designed to extract these signals robustly.
Who's Behind the Text? Robust Model Authorship Attribution
Minseok Ko, Hyunseok Lee, Woomin Song, Beomjun Kim, Jinwoo Shin, Kunwoong Kim
As large language models (LLMs) are increasingly used to generate text, their misuse has become a growing societal problem, with implications for academic misconduct, model distillation, and cyberattacks. For these reasons, Model Authorship Attribution (MAA), which identifies the source model of a given text, is important for promoting the responsible use of LLMs and improving AI safety. In practice, attribution systems must remain reliable beyond the distributions observed during training. They may encounter texts from unseen domains and writing settings, as well as generations from future versions of model families that were not available when the system was trained. However, existing MAA methods often exhibit substantial performance degradation under such distribution shifts. To address this challenge, we propose Authorship Tracing via Likelihood-based Attribution of Sources (ATLAS), which formulates MAA through source-conditioned text modeling rather than direct source prediction. ATLAS learns how text from each source family is generated and attributes a given text by comparing its likelihood under each learned source condition. We aggregate this likelihood evidence at the token level with clipping so that no single token dominates the decision. We further introduce MAABench, a broad MAA benchmark for evaluating practical OOD settings such as domain and version. Our experiments show that ATLAS consistently outperforms existing baselines across MAABench and OpenTuringBench. In particular, when both the generation domain and the model version change on MAABench, ATLAS achieves 74.27% accuracy, 9.00pp higher than the strongest baseline. Beyond MAA, ATLAS can also help identify distillation source models by measuring which source model a target LLM most closely resembles. Our extended experiments show that ATLAS recovers the known teacher of every distilled model we evaluate.
Superintelligence and the Triple Risk of an AI Race
Matthew Rendall, Max Hellrigel-Holderbaum, Leonard Dung
On many estimates, artificial general intelligence (AGI) is likely to emerge within the next few decades or even sooner. Once it does, it may quickly surpass human intelligence, possibly by a huge margin. Some believe that the country that develops AGI first could attain global dominance. This prospect risks setting off intense competition between the United States and China. Indeed, a narrative has already developed that the superpowers are locked in an AI race, with the Trump administration proclaiming its determination to 'dismantle unnecessary regulatory barriers' to winning it. We argue that this approach presents three profound dangers. The first two are familiar from past arms races: being overtaken by a rival or triggering a war. The third is new: losing control of human beings' future to agents far more intelligent than we are, which could endanger our very survival. Striving to get to AGI first risks a race to the bottom on safety practices and regulation concerning them. China does not yet appear bent on racing to superintelligence. The United States should prioritise both unilateral and co-operative efforts to improve AI safety and avoid provoking a race that all humanity could lose.
Trust Hinders Bidirectional Human-AI Alignment
Michaela Benk, Kacper Sokol
Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users exhibit appropriate reliance on them. We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support. To overcome this limitation, we develop three dimensions of AI-assisted decision making. First, trusting AI is only one of several possible strategies for managing uncertainty. Second, trust calibration is often misapplied when AI models "large-world" decision problems, i.e., ill-defined situations where goals and states are only partially known or unknowable. Third, in such settings human-AI alignment requires addressing higher-order decision processes. We conclude by reflecting on the implications of our observations on the design of process-oriented human-AI coupled systems that jointly identify and negotiate uncertainty throughout the decision-making task.
Distillation for Incrimination and Distillation for Capabilities
Sebastian Prasanna, Jacqueline Tay, Alek Westover
Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. However, distilling such a model into a weaker benign student places the teacher in a Distillation Double Bind: if misalignment transfers, the student may conceal it less effectively, revealing evidence about the teacher; if it does not, the student may learn useful capabilities while remaining benign. We introduce two distinct distillation approaches, one targeting each outcome. Distillation for Incrimination (DFI) aims to transfer misalignment but not the ability to conceal it. Distilling AuditBench's secret-keeping models into their underlying instruction-tuned model produces students that are significantly more likely than their teachers to admit their hidden behavior when asked, suggesting that knowledge of the behavior transferred more readily than the propensity to conceal it. Confession gains largely disappear when the student does not share the teacher's pretrained base, so DFI should target the teacher's own pre-RL checkpoint, which is weaker than the teacher but shares its base model. Distillation for Capabilities (DFC) aims to transfer capabilities but not misalignment. Among several techniques we evaluate, two are effective: inoculation prompting and training for more epochs on fewer unique examples. Both preserve the capability gains of standard distillation while substantially reducing the subliminal transfer of an animal preference, our proxy for misalignment. Together, these findings demonstrate two ways distillation can be used for AI safety: incriminating misaligned models, and extracting their capabilities without their misalignment.
Beyond the Ergodic Wall: A Discrete Geometric Physics Sandbox for Analysing AI Scaling Limits and Complexity Collapse
Simon Richard Daniel
This paper exposes the ergodic ceiling and thermodynamic inefficiency of current deep learning, which converges to a statistical average of historic human knowledge. True semantic novelty requires a path-dependent, spatiotemporally bounded observer (a Data LifeCone) to inject non-ergodic insight, achieving KL divergence and avoiding manifold lock-in. AI Safety must recognise that a mature Artificial Superintelligence (ASI) would regard human-AI symbiosis as a thermodynamic necessity to avoid model collapse. We therefore propose hard physical containment via a digital physics sandbox powered by a Holographic E8 Projection Engine to verify models against real-world constraints. Spacetime is modeled as an information substrate of nested face-centered cubic (FCC) lattices of oscillating Planck-scale spheres maximizing local information and entropy density. Cut-and-project methods from the E8 root lattice produce a quasi-crystalline geometry where tetrahedral voids support SU chiral structure and elastic-shear eigenvalues generate candidate mass spectra. Rest mass is treated as discrete, integer microstate counts on local holographic boundaries (Bekenstein bound), replacing floating-point approximations with strict integer arithmetic to provide an information-theoretic definition of matter. Stable particles emerge as recurring lattice dislocations, and continuum recovery proceeds via variational renormalisation-group flows and Fourier Neural Operators that learn continuous spectral operators to recover the Schrödinger equation as an emergent statistical description. Crucially, these top-down topological constraints offer a mechanism for "NP-to-P" complexity collapse: by restricting an algorithm's proposal space to physically conserved causal trajectories, the sandbox prunes the combinatorial tree to deterministic, polynomial-time paths.
Revisiting Explainable AI through Model-Independent Concept Dictionaries
Thomas Schnake, Doreen Schöppenthau, Alexander Meyer, Jacques Corbeil, Klaus-Robert Müller, Grégoire Montavon
Modern applications of AI rely on increasingly complex models. Explainable AI (XAI) has emerged as a set of techniques aimed at improving model transparency. However, existing XAI methods typically assume input features to be inherently interpretable, or they rely on intermediate internal abstractions that are difficult to characterize and highly architecture-specific, hindering consistent use across models. To address these limitations, we propose DictXAI, a method that defines concepts directly in the input domain via a dictionary---a large, potentially overcomplete set of predefined elements, each carrying an interpretable meaning. Technically, DictXAI first computes a sparse code of the input and then attributes the model's prediction to the associated dictionary elements. We demonstrate the actionable nature of DictXAI explanations, showing that they can attribute AI malfunctions (e.g., Clever Hans effects) directly to identifiable artifact patterns in the data, while fostering human-AI alignment on intricate biomedical signals. We further demonstrate our method's ability to operate across a wide variety of dictionaries, including learned image bases, analytically defined waveforms for electrocardiography, and experimentally acquired dictionary elements. Overall, our results show that DictXAI provides more interpretable, actionable, and architecture-agnostic insights than classical XAI or existing concept-based approaches.
AI Safety Considerations for Agents With Limited Time to Act
Leo Zeitler, Jack Richings, Victoria Nockles
In the wake of the increasingly public discussion about AI alignment, recent work has tried to propose specific AI architectures that behave safely. However, the proposed arguments that seemingly demonstrate proved alignment mostly neglect the environment the agent needs to act in. We discuss theoretical bounds for agent-agnostic safety guarantees in environments that can only be partially observed and within which an action is required within limited time. We introduce two realistic scenarios, one with an infinite state space and one with signal mixture. In these scenarios, we prove that even a perfect agent cannot guarantee safe behaviour. It will be argued that for any proof of AI safety or alignment, the environment and associated safe actions need to be specifically considered together with the agent.
Foundational papers
- 2023Ai alignment: A comprehensive survey763 citations
- 2025AI Safety, Alignment, and Ethics (AI SAE)3 citations
- 2026Legal alignment for safe and ethical ai24 citations
- 2026-10
Ai alignment: A comprehensive survey
J Ji, T Qiu, B Chen, B Zhang, H Lou, K Wang et al.
… Specifically, we survey assurance methods of AI systems throughout their lifecycle, covering safety evaluation, interpretability, and human value compliance. We discuss current and …
Helpful, harmless, honest? Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human Feedback: AD Lindström et al.
A Dahlgren Lindström, L Methnani, L Krause et al.
… and effectiveness of the dominant performance evaluation criteria, as well as of the safety claims made about the widely-used RLHF approaches to AI alignment and safety today. …
The landscape of ai alignment: A comprehensive review of theories and methods
X Li, Q Jiang, L Jiang, S Zhang, S Hu
… AI alignment, aiming to provide a comprehensive understanding of its fundamental concepts, motivations, and the alignment … growing body of knowledge in AI safety, offering insights for …
Legal alignment for safe and ethical ai
N Kolt, N Caputo, J Boeglin, C O'keefe et al.
… of ensuring AI systems comply with those specifications. To date, AI alignment has generally … In this paper, we survey the emerging field of legal alignment that aims to fill this gap and …
The frontier of AI alignment: challenges and strategies for future ai systems
T Duenas, D Ruiz
… It highlights the importance of implementing strict safety … of AI alignment must not only focus on developing more sophisticated alignment techniques but also on creating robust safety …
AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?
L Dung, F Mai
… AI alignment research aims to develop techniques to ensure that AI systems do not cause … understanding the current level of risk and how to prioritize AI alignment research in the future. …
The many faces of AI alignment
A Kasirzadeh
… This might involve proving partial alignment properties or safety bounds under more … We can now see that these positions often concern different cells of the AI alignment problem …
Disentangling AI alignment: a structured taxonomy beyond safety and ethics
K Baum
… Rather than proposing another high-level theory or technical alignment approach, … AI alignment. Starting from parameterized notions of safety and ethicality as special cases of alignment…
AI Safety, Alignment, and Ethics (AI SAE)
D Waldner
This paper grounds ethics in evolutionary biology, viewing moral norms as adaptive mechanisms that render cooperation fitness-viable under selection pressure. Current alignment …
On AI operational alignment and misalignment for high-risk AI systems: case study
R Mihai, B Daireaux, E Cayeux
… AI alignment is generally associated with ethics and social aspects. For high-risk AI systems, principles such as safety, … By safety it is meant both safety of the system itself but also …
How this page is made. Foundational papers and citation counts come from academic search via the AISA Scholar API, with abstracts read from each paper’s page; recent preprints and monthly volumes come from the public arXiv API (metadata CC0). The digest is written by a language model from those abstracts only — every theme links the papers it draws on, and figures not in the inputs are removed.
Sources & citations
- Ai alignment: A comprehensive survey 763 citations
- Helpful, harmless, honest? Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human Feedback: AD Lindström et al. 96 citations
- The landscape of ai alignment: A comprehensive review of theories and methods 5 citations
- Legal alignment for safe and ethical ai 24 citations
- The frontier of AI alignment: challenges and strategies for future ai systems 8 citations
- AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures? 6 citations
- The many faces of AI alignment 3 citations
- Disentangling AI alignment: a structured taxonomy beyond safety and ethics 7 citations
- AI Safety, Alignment, and Ethics (AI SAE) 3 citations
- On AI operational alignment and misalignment for high-risk AI systems: case study 1 citations
- Evaluating Model Capabilities After Benchmark Saturation arXiv · 2026-10-14
- Who's Behind the Text? Robust Model Authorship Attribution arXiv · 2026-10-14
- Superintelligence and the Triple Risk of an AI Race arXiv · 2026-10-12
- Trust Hinders Bidirectional Human-AI Alignment arXiv · 2026-10-10
- Distillation for Incrimination and Distillation for Capabilities arXiv · 2026-10-07
- Beyond the Ergodic Wall: A Discrete Geometric Physics Sandbox for Analysing AI Scaling Limits and Complexity Collapse arXiv · 2026-10-07
- Revisiting Explainable AI through Model-Independent Concept Dictionaries arXiv · 2026-10-07
- AI Safety Considerations for Agents With Limited Time to Act arXiv · 2026-10-07