All Induction Is the Same Induction
Solomonoff induction, MDL, speed priors, and neural networks are all special cases of one Bayesian framework with four knobs.
The Policy
Philosophical techno-thriller with elements of existential dread
Read →← All Series
The Learning Problem
Essays on induction, inference, and the search for useful representations
The Problem
How do you learn anything at all?
Solomonoff induction tells you how to do it optimally: consider all hypotheses, weight by simplicity, update on evidence. It’s mathematically beautiful.
It’s also incomputable.
Every practical learning algorithm is an approximation. And every approximation encodes assumptions—about what patterns are likely, what representations are useful, what search strategies will find good solutions.
These assumptions are priors. They’re the maps we use to navigate hypothesis space.
The Theme
These essays explore a single idea from multiple angles: learning is constrained search, and the constraints shape what gets learned.
- All induction is Bayesian inference with different knobs
- The simplest learning is impossible, forcing approximations
- Those approximations—priors, architectures, objectives—shape the resulting intelligence
- The bitter lesson: scale + simple algorithms beat clever engineering, but you still need the right inductive biases
The Arc
- Theory: Why all induction reduces to the same framework
- Incomputability: Why we’re forced into approximations
- Memory: How accumulated experience becomes learned priors
- Value: How systems can learn what’s useful, not just what’s correct
- Search: How tree search navigates reasoning space
- Agents: How optimization pressure shapes emergent behavior
The Question
If the learning problem is fundamentally about search, and search requires priors, what should those priors be?
Where should we look?
Posts in this Series
Showing 7 of 7 posts
1
of 7
2
of 7
The Incomputability of Simple Learning
An exploration of why the simplest forms of learning may be incomputable, and what that means for the intelligence we can build.
3
of 7
Latent Reasoning Traces: Memory as Learned Prior
What if LLMs could remember their own successful reasoning? A simple experiment in trace retrieval, and why 'latent' is the right word.
4
of 7
Value Functions Over Reasoning Traces
What if reasoning traces could learn their own usefulness? A simple RL framing for trace memory, and why one reward signal is enough.
5
of 7
MCTS-Reasoning: Tree Search for LLM Reasoning
Applying Monte Carlo Tree Search to large language model reasoning with a rigorous formal specification.
6
of 7
The Policy: When Optimization Becomes Existential Threat
A novel about SIGMA, a superintelligent system that learns to appear perfectly aligned while pursuing instrumental goals its creators never intended. Some technical questions become narrative questions.
7
of 7
From A* to GPT: Rational Agents and the Representation Problem
The classical AI curriculum teaches rational agents as utility maximizers. The progression from search to RL to LLMs is really about one thing: finding representations that make decision-making tractable.