Prediction

A Formal Theory of Inductive Inference (Parts I & II)

Notes

Foundational paper on algorithmic probability and universal induction. Basis for AIXI.

N-gram Language Models

The classical approach to sequence prediction—counting and smoothing

Data Generating Processes for Sequential Prediction

Markov processes and tree sources—understanding where sequences come from

The Bayesian Prediction Framework

Model averaging over hypotheses—the principled approach to handling uncertainty in prediction

Solomonoff Induction: The Theoretical Ideal

Why the optimal predictor is incomputable—and what we can learn from it anyway

Introduction to Sequential Prediction

The fundamental problem of predicting what comes next—from compression to language models