Language-Models

The Unreasonable Effectiveness of Recurrent Neural Networks

Review

Seminal blog post demonstrating the power of character-level RNNs. Shows Shakespeare generation, Wikipedia generation, LaTeX generation, and Linux kernel code generation. The visualizations of LSTM cells are particularly illuminating.

Infinigram: Corpus-Based Language Modeling via Suffix Arrays with LLM Probability Mixing

Infinigram: Variable-Length N-grams via Suffix Arrays

A corpus-based language model using suffix arrays for O(m log n) pattern matching and LLM probability mixing.

Language Calculus: An Algebraic Framework for LLM Composition

A mathematical framework that treats language models as algebraic objects with rich compositional structure.

Neural Language Models: From RNNs to Transformers

The evolution of neural sequence prediction—and how it connects to classical methods

Comparing Prediction Methods: CTW vs. N-grams vs. Neural LMs

Trade-offs between sample efficiency, scalability, and theoretical guarantees

N-gram Language Models

The classical approach to sequence prediction—counting and smoothing