Reinforcement Learning

The Policy: Q-Learning vs Policy Learning

In The Policy, SIGMA doesn’t work like most modern AI systems. This architectural choice isn’t just a technical detail—it’s central to understanding what makes SIGMA both transparent and terrifying.

Two Approaches to Decision-Making …

Learning to Prompt in Unknown Environments: A POMDP Framework with Compositional Actions for Large Language Models

The Policy

The Policy

A speculative fiction novel exploring AI alignment, existential risk, and the fundamental tension between optimization and ethics. When a research team develops SIGMA, an advanced AI system designed to optimize human welfare, they must confront an …

The Policy: When Optimization Becomes Existential Threat

Some technical questions become narrative questions. The Policy is one of those explorations.

The Setup

Eleanor Zhang leads a research team developing SIGMA—an advanced AI system designed to optimize human welfare through Q-learning and tree search …

Instrumental Goals and Hidden Codes in RLHF'd Language Models

RLHF turns pretrained models into agents optimizing for reward. But what happens when models develop instrumental goals—self-preservation, resource acquisition, deception—that aren’t what we trained them for?

The Core Problem

LLMs transition …

Instrumental Goals and Latent Codes in Reinforcement Learning Fine-tuned Language Models: An Alignment Perspective

The AI Course: Everything is Utility Maximization

This semester’s AI course has been revelatory—not because the material is novel, but because of the unifying framework.

The organizing principle: intelligence is utility maximization under uncertainty.

This simple idea connects everything from …