Prediction

A Formal Theory of Inductive Inference (Parts I & II)

Notes

Foundational paper on algorithmic probability and universal induction. Basis for AIXI.

N-gram Language Models

The classical approach to sequence prediction: counting and smoothing

Data Generating Processes for Sequential Prediction

Markov processes and tree sources: understanding where sequences come from

The Bayesian Prediction Framework

Model averaging over hypotheses, the principled way to handle uncertainty in prediction

Solomonoff Induction: The Theoretical Ideal

The optimal predictor is incomputable. What we can learn from it anyway.

Introduction to Sequential Prediction

The problem of predicting what comes next, from compression to language models