Deep Manifold Interpretation

2026 Q2 Collection

Like many others, I read AI papers on a very frequent basis. In early May, I started the 'Deep Manifold Interpretation' series as a way to examine the breadth and depth of the framework. People often ask me how to start learning about Deep Manifold, and reading these interpretations is an excellent way to begin.

June, 2026

1. Novel Idea and Loss in LLM

2. AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

3. Efficient Pre-Training with Token Superposition

4. Greed Is Learned: Visible Incentives as Reward-Hacking Triggers

5. Scalable Circuit Learning for Interpreting Large Language Models

6. Can LLM Coding Agents Reason About Time Series?

7. Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

8. Persistent Topological Features in Large Language Models

9. Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers

10. Pre-training under Infinite Compute

11. Nothing from Something: Can a Language Model Discover 0?

12. Emergent Analogical Reasoning in Transformers

13. Artificial Jagged Intelligence: When AI Benchmarks Misstate Deployment Value

14. Preserving Plasticity in Continual Learning via Dynamical Isometry

15. The Cheap, Intelligent, Fast (CIF) Theorem for AI Models

16. From AGI to ASI

17. A Bitter Lesson for Data Filtering

18. DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation (Sakana AI)

19. MiniMax Sparse Attention

20. FireAct: Toward Language Agent Fine-tuning

21. Recursive Self-Improvement bounded research (Recursive AI)

22. Topological Neural Operators

23. Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

24. Do Transformers Need Three Projections? Systematic Study of QKV Variants

25. Specialization of softmax attention heads: insights from the high-dimensional single-location model

26. Pruning and Distilling Mixture-of-Experts into Dense Language Models

27. Training-Free Layout Control with Cross-Attention Guidance

28. Topological Neural Operators

29. A Model of the Language Process

30. On the Geometry of On-Policy Distillation

31. Efficient Mean Curvature Computation on High-Dimensional Data Manifolds

32. Principles and Practice of Deep Representation Learning (Yi Ma book)

33. Large Language Models Hack Rewards, and Society

34. If LLMs Have Human-Like Attributes, Then So Does Age of Empires II

35. Stochastic Perturbations Improve Distribution-to-Distribution Generative Models

36. Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic Artificial Intelligence

37. Oscillatory State-Space Models as Inductive Biases for Physics-Informed Neural PDE Solvers

38. Subliminal Effects in Your Data: A General Mechanism via Log-Linearity

39. Edge of Stability Selectively Shapes Learning Across the Data Distribution

40. How Attentive Are Graph Attention Networks?

41. Spectral Scaling Laws of Muon

42. Procedural Memory Distillation: Online Reflection for Self-Improving Language Models

43. Critiques of World Models

44. Subliminal Effects in Your Data- A General Mechanism via Log-Linearity

45. A Functional Taxonomy of World Models (Fei-Fei Li)

46. Self-Improving Language Models with Bidirectional Evolutionary Search

47. SkillOpt: Executive Strategy for Self-Evolving Agent Skills

48. Beyond Objects: Contextual Synthetic Data Generation for Fine-Grained Classification

49. Self Improving Language Models with Bidirectional Evolutionary Search

50. Rethinking Memory as Continuously Evolving Connectivity

51. From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models

52. Don’t be lazy: CompleteP enables compute-efficient deep transformers

53. Parallax: Parameterized Local Linear Attention for Language Modeling

54. MAI-Thinking-1: Building a Hill-Climbing Machine (Microsoft)

55. Geometry-Aware Contrastive Learning for Few-Shot Automatic Modulation Recognition

56. Trust-Region Behavior Blending for On-Policy Distillation

57. Lattice Deduction Transformers

58. Targeted Neuron Modulation via Contrastive Pair Search

59. Neural Weight Norm = Kolmogorov Complexity

60. Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention

May 2026

1. Learn from your own latents and not from tokens: A sample-complexity theory

2. DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking

3. Categorical Deep Learning is an Algebraic Theory of All Architectures

4. NOBLE nonlinear low-rank adaptation

5. Recursive Language Models

6. Tokenisation via Convex Relaxations

7. Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence

8. Epicure: Navigating the Emergent Geometry of Food Ingredient Embeddings

9. PowLU: An Activation Function for Stable Pre-Training of LLMs

10. Emergent Introspective Awareness in Large Language Models

11. ECHO: Terminal Agents Learn World Models for Free

12. Simulation to Enaction: Post-trained Language Models Recognize and React to their own Generations

13. Deep sequence models tend to memorize geometrically; it is unclear why

14. LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws

15. The Rules of the Game: A Survey of Rubrics for Large Language Models

16. NITP: Next Implicit Token Prediction for LLM Pre-training

17. Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units

18. Data Value Density Enhancement for Large Language Model Training: A Comprehensive Survey

19. Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections

20. Characterizing, Evaluating, and Optimizing Complex Reasoning

21. MeMo: Memory as a Model

22. Hallucinations Undermine Trust; Metacognition is a Way Forward

23. Generative Recursive Reasoning

24. HRM-Text: Efficient Pretraining Beyond Scaling

25. What do Language Models Learn and When? The Implicit Curriculum Hypothesis

26. Probabilistic Tiny Recursive Model

27. Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions

28. Steered LLM Activations are Non-Surjective

29. Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling

30. Muon is Not That Special: Random or Inverted Spectra Work Just as Well

31. We finally uncovered what AIs REALLY think.

32. The Truth Lies Somewhere in the Middle (of the Generated Tokens)

33. Neural networks vs polynomial approximations (Runge phenomenon)

34. Positive Alignment: Artificial Intelligence for Human Flourishing

35. A Theory of Generalization in Deep Learning

36. Pixel-space generation is not representation-free

37. Straight lines belong to man, the curved ones to God.

38. Mechanistic Interpretability be like..

39. LLMs are just stacks of Attn+MLP

40. Sparsity as the way out