Bernoulli Types

Two-Sided Error in Approximate Membership,
Retrieval, and Computation

Alexander Towell

Southern Illinois University Edwardsville

October 2026

Bernoulli Types: Two-Sided Error in Approximate Membership, Retrieval, and Computation

© 2026 Alexander Towell.

This work is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). You may share and adapt it for any purpose, including commercially, provided you give appropriate credit, link to the license, and indicate if changes were made. License text: https://creativecommons.org/licenses/by/4.0/.

Source code embedded in this work (listings and examples) and the companion Python package bernoulli-types are licensed under the MIT License.

Drafting and verification of this work were assisted by AI tools (Anthropic Claude and OpenAI Codex). The author is responsible for all of its content, including any errors.

This edition: October 2026.

Author: Alexander Towell, lex@metafunctor.com
ORCID: 0000-0001-6443-9897
Web: https://metafunctor.com
Source: https://github.com/queelius/bernoulli/tree/main/book

Contents

List of Figures

Preface

This book is about one object. A query returns a bit, and the bit is observed through a binary channel: a true negative is reported as positive with probability \(\fprate \) (the false-positive rate) and a true positive is reported as negative with probability \(\fnrate \) (the false-negative rate). A Bloom filter answering a membership query is such a channel, with \(\fnrate = 0\). So is one round of the Miller–Rabin test, a survey respondent answering under randomized response, a noisy classifier, the readout of a qubit, and a keyword lookup in a Bloom-filter-based encrypted index. The book takes the channel as its primitive and follows it outward: to Boolean combinations of noisy bits, to approximate sets, maps, relations and algebraic types, to constructions that meet the space bounds, and finally to the places where the model stops describing the system in front of it.

What kind of contribution this is

The contribution is a unification, not a collection of new theorems. Nearly every individual result in these pages has a named predecessor, and several are decades old. The per-element channel is randomized response on an indicator [War65]. The \(\log _2(1/\fprate )\) bits-per-element bound for approximate membership is due to Carter et al. [Car+78], and the case with false negatives was settled to leading order by Pagh and Rodler [PR01]. Forward propagation of error probabilities through logic gates goes back at least to Parker and McCluskey [PM75], and the noisy-parity identity is Lemma 1 of Gallager [Gal62]. The perfect hash filter, a minimal perfect hash function plus short fingerprints, is described by Dietzfelbinger and Pagh [DP08]. Program values that carry a correctness probability through a computation are the subject of Decaf [Bos+15] and its relatives, and a probabilistic relational algebra was given by Fuhr and Rölleke [FR97]. Each chapter ends with a section titled Notes and Lineage that says who established each main result, what the chapter adds (usually a presentation, a unification, or a correction), and what it deliberately leaves out. The intent is that no reader should have to guess whether a result is being claimed.

What the unified treatment does add, I think, is twofold. First, a single vocabulary (the rate pair, model order, the Kronecker factorization of independent queries) for objects that live in communities which rarely cite one another: data structures, databases, survey statistics, diagnostic testing, complexity theory, cryptography, and quantum measurement. Second, an honest map of where that vocabulary fails. The most important entry on the map is in Chapter 3: a rate pair does not propagate through a Boolean formula. The false-positive rate of an AND depends on which negative input assignment occurred, so a gate that receives only a summary pair can be wrong by a factor of two on a three-input formula (Example 3.1). What does propagate exactly is the error conditional on each latent input assignment; prior-averaged rates and worst-case bounds are the honest summaries. Shared inputs, shared intermediate values, adaptive queries and construction-time conditioning each break the independence the model needs, and Part V is devoted to that boundary.

Relation to the source manuscripts

The book grew out of a set of manuscripts on Bernoulli sets, their composition, classification measures, entropy bounds, the Bernoulli hash function, the perfect hash filter, approximate maps, approximate relations, algebraic types, and quantum readout [Tow26b; Tow26f; Tow26c; Tow26g; Tow26j; Tow26m; Tow26i; Tow26h; Tow26k; Tow26a]. Writing the book, and auditing it with exhaustive finite enumeration and exact rational arithmetic, turned up errors in several of them: a claim that the rate pair carries through an arbitrary Boolean tree; a finite order-statistic law for the Bernoulli hash function that fails already at two keys and two slots; an adaptive acceptance threshold presented as saving space when it trades away recall; a closure theorem for composed approximate maps that fails when two inputs share an intermediate value; overbroad closure and independence claims for approximate relations; a scope statement for quantum readout that confused a diagonal measurement with a diagonal state; and a space lower bound with false negatives described as open when it is not. Earlier drafts of this book had their own errors, among them a perfect-hash-addressed fingerprint table presented as meeting the space floor, which no minimal perfect hash permits. Where a chapter departs from a source manuscript, the chapter is the corrected version and supersedes it; Chapter B tabulates the corrections.

Intended reader

The reader I have in mind is a researcher or graduate student who already works with one of these objects (a filter, a sketch, a diagnostic test, a randomized algorithm, a probabilistic database, an encrypted index, a readout model) and wants to see it in relation to the others. The book assumes undergraduate probability, basic linear algebra, and comfort with proofs. Information theory beyond entropy and channel capacity is developed where it is used. The book is not a survey of filter engineering and reports no performance benchmarks; for the current state of practical filters and retrieval structures see Graf and Lemire [GL22] and Dietzfelbinger et al. [Die+26].

How to read it

Part I (Chapters 1 to 4) builds the atom: the noisy bit, binary channels and model order, Boolean algebra under noise with the latent-assignment caveat, and the two independence axioms with the Kronecker factorization they imply. Everything later depends on it, Chapters 3 and 4 most of all. Part II (Chapters 5 to 8) moves from bits to sets: the Bloom filter read as a channel, set operations and the model order they raise, classification measures as random variables, and the space bounds. Part III (Chapters 9 to 13) generalizes to maps and \(n\)-ary channels, sketches, randomized algorithms, error contracts for approximate relations, and algebraic types. Part IV (Chapters 14 and 15) treats constructions that approach the bounds, read as seed search and as perfect hashing with fingerprints. Part V (Chapters 16 to 19) is the boundary: where the model fails, cipher maps and trapdoor computing, encrypted search, and the operator representation that connects the framework to quantum measurement.

Readers with narrower aims can take shorter paths. For data structures: Part I, Chapters 5, 8 and 9, and Part IV. For databases: Part I, Chapters 6 and 12. For security: Part I, Chapter 5, and Chapters 16 to 18. For quantum readout: Chapters 2 to 4, Chapter 16, and Chapter 19. Chapter A collects the notation, Chapter B restates the framework’s principles together with their scope, and Chapter C indexes the companion code.

The companion package and notebooks

The Python package bernoulli-types (import name bernoulli), maintained in the bernoulli-py/ directory of the source repository, is the book’s executable companion. It implements the atom, exact error propagation for Boolean formulas over independent noisy bits, approximate sets, maps, relations and sketches, and the classification measures. Fifteen notebooks reproduce the numerical claims of the chapters they accompany; the chapters without a notebook (Chapters 4, 13, 17 and 19) are theoretical. The package is research software at an alpha stage, not a production library.

Names

“Bernoulli” is overloaded. A Bernoulli filter in target tracking [Ris+13], a Bernoulli factory in simulation [KO94], and the Bernoulli map of dynamical systems are unrelated to the objects here. In this book the name refers only to the Bernoulli-distributed error indicators that the framework attaches to each query.

License and acknowledgment

The text is released under the Creative Commons Attribution 4.0 International License (CC BY 4.0); embedded code and the companion package are under the MIT License. Drafting and verification were assisted by AI tools (Anthropic Claude and OpenAI Codex), used for drafting prose, checking claims by exhaustive enumeration, auditing them against the literature, and writing companion code. That assistance is not a substitute for a reader’s scrutiny: earlier review rounds, human and machine, passed errors that later audits caught. I am responsible for all of its content. Corrections are welcome and will be recorded in the source repository.

Alexander Towell
October 2026

Part I
The Atom

Chapter 1
The Noisy Bit

1.1 Three Faces of a Noisy Bit

Three systems from different corners of applied probability each produce a single Boolean output that may be wrong, and at the level of input-output behavior they are one object. We introduce them as running examples and extract that object.

The biased coin

Consider a coin whose bias depends on how it is held: palmed “in” it lands heads with probability \(\tau \), palmed “out” with probability \(\epsilon \), where \(\tau \) and \(\epsilon \) are neither equal nor near \(1/2\). Read “in” as a positive condition and “out” as a negative one. The coin reports each with its own reliability: two inputs, one Boolean output, two independent parameters. It returns as an asymmetric channel in Section 2.2 and as the worked instance of the formal definition in Chapter 4.

The binary symmetric channel

A single bit crosses a noisy line, a wire, a wireless link, a chip-to-chip trace. For each bit independently the line flips the symbol with crossover probability \(\fprate \) and passes it through otherwise, with the same \(\fprate \) in both directions: a \(\top \) flips to \(\bot \) and a \(\bot \) to \(\top \) with equal probability. This is the binary symmetric channel (BSC), the simplest model of a noisy one-bit line and the most studied channel in information theory; the term channel is Shannon’s, for any single-input, single-output stochastic transmission. The symmetry is a real constraint, since many physical channels are asymmetric and need a separate rate per direction (Section 2.2). Chapter 2 gives the BSC a formal definition and a matrix representation (Section 2.1).

The classifier verdict

A spam filter emits a verdict on each message, spam (\(\top \)) or ham (\(\bot \)). It flags ham as spam at the false-positive rate \(\fprate \) and misses spam at the false-negative rate \(\fnrate \), and these are generally unequal: a filter that drops legitimate mail at \(1\%\) is a different product from one that leaks \(1\%\) of spam. Here the two parameters are forced by the problem rather than by happenstance, since the two errors are distinct harms with distinct costs. A score-thresholded classifier traces a curve of \((\fprate , \fnrate )\) pairs as its threshold sweeps, the trade-off the ROC curve re-axes; no threshold drives both rates to zero at once.

The shared object

A coin held in or out, a wire carrying one bit, a filter emitting one verdict: physical, informational, and algorithmic, yet identical in input-output behavior. Each maps a Boolean input to a Boolean output that is correct with some probability and wrong otherwise, and the error behavior is fixed by at most two numbers, the rate of reporting \(\top \) on a \(\bot \)-input and the rate of reporting \(\bot \) on a \(\top \)-input. The BSC carries one parameter because its two rates are constrained equal; the coin and the filter carry two. Two numbers describe everything probabilistic at the single-query level. We call this object a noisy bit. The object is not new: it is the asymmetric binary channel of information theory, the misclassification model of the \(2 \times 2\) table, and, when the noise is added on purpose to protect a respondent, randomized response (see the Notes and Lineage at the end of this chapter). What the book adds is the decision to treat these as one atom and to build everything else from it.

The three faces used three vocabularies because each is natural in its own letters, and they reconcile directly: the coin’s \(\tau \) is the true-positive rate \(\tprate = 1 - \fnrate \) and its \(\epsilon \) is \(\fprate \); the classifier’s \((\fprate , \fnrate )\) are the same quantities; the BSC is the special case \(\fprate = \fnrate \). Three commitments follow and hold throughout Part I. The noisy bit is atomic: one Boolean input slot, one Boolean output slot, nothing smaller, and everything larger (multi-bit channels, repeated-trial algorithms such as Miller-Rabin, classifier ensembles) is built from noisy bits, as chapters 2 and 4 show. It is behavioral: the description fixes the rates and says nothing about mechanism, so a biased coin, a thermal photon detector, and a logistic-regression classifier with the same rates are the same noisy bit. And it is parsimonious: at most two numbers per noisy bit. The rest of Part I is the algebra of how these atoms compose.

1.2 Quantifying the Noise

A noisy bit is fixed by two conditional error rates. We make “rate” precise through the \(2 \times 2\) contingency table, name the four rates it yields, and relate them to the precision and recall of binary classification.

The contingency table

Run a noisy bit \(n\) times. Each trial has a known true input, \(\top \) or \(\bot \), and an observed output, so the \(n\) trials are ordered pairs (truth, output), each falling into one of four joint states: a true positive (truth \(\top \), output \(\top \)), a false negative (truth \(\top \), output \(\bot \)), a false positive (truth \(\bot \), output \(\top \)), and a true negative (truth \(\bot \), output \(\bot \)). The term “false negative” names the output, not the truth: a \(\bot \) reported when the truth was \(\top \). Writing TP, FN, FP, TN for the counts, so that \(\mathrm {TP} + \mathrm {FN} + \mathrm {FP} + \mathrm {TN} = n\), arrange them by truth (rows) and observed output (columns):

Rates

The two error rates are conditional, each normalized by its truth row: \[ \fprate \;=\; \frac {\mathrm {FP}}{\mathrm {TN} + \mathrm {FP}}, \qquad \fnrate \;=\; \frac {\mathrm {FN}}{\mathrm {TP} + \mathrm {FN}}. \] Dividing by the row total rather than by \(n\) is what makes \(\fprate \) and \(\fnrate \) properties of the noisy bit itself rather than of the input mixture observed. The complementary rates are the true-negative rate \(\tnrate = 1 - \fprate \) (the \(\bot \)-inputs reported correctly, also specificity) and the true-positive rate \(\tprate = 1 - \fnrate \) (also sensitivity, and recall in the vocabulary below). The two free parameters of a noisy bit are \(\fprate \) and \(\fnrate \); the other two are derived. The biased coin’s \((\tau , \epsilon )\) of Section 1.1 is \((\tprate , \fprate )\), and the binary symmetric channel of Section 2.1 is the constrained case \(\fprate = \fnrate \), where one parameter suffices. These four symbols keep these meanings throughout the book.

Precision and recall

A practitioner of binary classification reads Table 1.1 through precision and recall: \[ \mathrm {precision} \;=\; \frac {\mathrm {TP}}{\mathrm {TP} + \mathrm {FP}}, \qquad \mathrm {recall} \;=\; \frac {\mathrm {TP}}{\mathrm {TP} + \mathrm {FN}}. \] Recall coincides with the true-positive rate, \(\mathrm {recall} = \tprate = 1 - \fnrate \). Precision does not reduce to the rates of the noisy bit: its denominator is a column total, which depends on how many \(\top \)- and \(\bot \)-inputs the sample contained. With \(n_\top \) positives and \(n_\bot \) negatives, \(\mathrm {TP} \approx n_\top \tprate \) and \(\mathrm {FP} \approx n_\bot \fprate \), so \[ \mathrm {precision} \;\approx \; \frac {n_\top (1 - \fnrate )}{n_\top (1 - \fnrate ) + n_\bot \fprate }. \] Precision is therefore a function of the prevalence \(n_\top / n_\bot \) as well as the rates, whereas \((\fprate , \fnrate )\) is intrinsic to the noisy bit. The pair \((\fprate , \fnrate )\) describes the noisy bit; the pair (precision, recall) describes the noisy bit together with a workload. This is why the framework takes \((\fprate , \fnrate )\) as the primary description and treats precision and accuracy as derived; Chapter 7 develops the derived measures as random variables.

An instance

A filter run on \(n = 100\) messages, 40 spam (\(\top \)) and 60 ham (\(\bot \)), produces the table

with rates \(\fprate = 3/60 = 0.05\) and \(\fnrate = 5/40 = 0.125\), hence \(\tnrate = 0.95\) and \(\tprate = 0.875\). Recall is \(35/40 = 0.875 = \tprate \), while precision is \(35/38 \approx 0.921\), exceeding \(\tprate \) because the column total (38) falls short of the truth row total (40). At a different prevalence the same \((\fprate , \fnrate )\) would yield a different precision.

1.3 A First Look at the Boolean Universe

A noisy bit has two inputs and two outputs, so its single-trial behavior is exhausted by four input-output combinations. Section 1.2 arranged these as a table of counts; abstracted from any particular run they are a table of outcome types, indexed by true input (rows) and observed report (columns):

The four cells carry the same labels as the contingency table of Section 1.2, TP, FN, FP, TN, seen from a different angle: the contingency table counts how many of \(n\) trials land in each cell, while the outcome matrix names the cells themselves. The empirical table is a sample from the joint distribution over these cells.

The smallest non-trivial universe

The input domain \(\{\top , \bot \}\) has exactly two elements. We call this set the Boolean universe and write it \(\Universe = \{\top , \bot \}\), borrowing universe from set-theoretic usage as the ambient set over which subsets are formed and predicates evaluated. It is the smallest universe that supports a non-trivial predicate: one element admits no discrimination, two already admit the question whether an input is one value or the other. Part I develops the algebra of this universe and nothing larger. Every richer universe (multi- bit inputs, finite enumerations, arbitrary algebraic types) is assembled from Booleans by Kronecker products (Section 4.4) and their lifts in Part III, so the algebra of these four cells governs systems that look more complicated but reduce, on inspection, to compositions of noisy bits. The reduction is exact per input; chapter 3 shows that what a composite inherits is a description indexed by the latent inputs, not a single pair of rates (Section 3.3).

The deferred joint distribution

A fully formal description assigns the four cells a joint distribution: given the rates of Section 1.2 and the prevalence of \(\top \) in the input stream, the cells receive four probabilities summing to one. We defer that step. The formal object whose instances are exactly the noisy bits with two free rate parameters is defined in Section 4.5, which earns the joint distribution by first stating the two independence axioms that make the type well-posed and then proving the Kronecker factorization that justifies the parameter economy. Writing the joint mass here would either duplicate that machinery or assert it without support. Definition 4.5.1 gives the joint table, the channel matrix, and the worked biased-coin instance.

For one concrete instance, take the biased coin with \(\fprate = 0.1\) and \(\fnrate = 0.2\). Of its \(\top \)-trials a fraction \(\tprate = 0.8\) land in the correct-positive cell and \(\fnrate = 0.2\) in the false-negative cell; of its \(\bot \)-trials a fraction \(\tnrate = 0.9\) land in the correct-negative cell and \(\fprate = 0.1\) in the false-positive cell. These are conditional shares within each row; converting them to outright cell probabilities requires the prevalence of \(\top \), which is exactly the joint distribution deferred to Section 4.5. The next section estimates the rates from data without waiting for that formalization.

1.4 Sampling and Estimating Rates

In practice the rates are not given: a classifier arrives without a label stating its false-positive rate, and a channel must be characterized from observed traffic. We state how to estimate \(\fprate \) and \(\fnrate \) from a finite sample and how to quantify the uncertainty. The full theory of these estimators (bias, variance, efficiency, and interval forms) is developed in Chapter 7.

Binomial estimators

Run a noisy bit \(n\) times, with \(n_0\) trials of truth \(\bot \) and \(n_1 = n - n_0\) of truth \(\top \). Each \(\bot \)-trial reports \(\top \) independently with probability \(\fprate \), so the false-positive count is binomial, \(\mathrm {FP} \sim \mathrm {Binomial}(n_0, \fprate )\), and likewise \(\mathrm {FN} \sim \mathrm {Binomial}(n_1, \fnrate )\). The row-conditional proportions \[ \hat {\fprate } = \frac {\mathrm {FP}}{n_0}, \qquad \hat {\fnrate } = \frac {\mathrm {FN}}{n_1} \] are unbiased, \(\mathbb {E}[\hat {\fprate }] = \fprate \), with variances \[ \mathrm {Var}(\hat {\fprate }) = \frac {\fprate (1 - \fprate )}{n_0}, \qquad \mathrm {Var}(\hat {\fnrate }) = \frac {\fnrate (1 - \fnrate )}{n_1}. \] The variance shrinks as \(1/n_0\), and its numerator \(\fprate (1 - \fprate )\) peaks at \(\fprate = 1/2\), where the noisy bit is hardest to characterize, and vanishes at \(\fprate \in \{0, 1\}\), where one trial settles the matter.

A confidence interval

For large \(n_0\) the central limit theorem makes \(\hat {\fprate }\) approximately Normal with mean \(\fprate \) and variance \(\fprate (1 - \fprate )/n_0\). Substituting the estimate into the variance gives the Wald interval at level \(1 - \alpha \), \[ \hat {\fprate } \;\pm \; z_{\alpha /2} \sqrt {\frac {\hat {\fprate }(1 - \hat {\fprate })}{n_0}}, \] with \(z_{\alpha /2}\) the upper-\(\alpha /2\) standard-Normal quantile. The Wald form is asymptotically correct but under-covers at small \(n_0\) when \(\fprate \) is near \(0\) or \(1\), and can even leave \([0,1]\); the Wilson, Clopper-Pearson, and Jeffreys intervals handle the boundary better, and Chapter 7 develops them.

Reproducibility

A companion notebook reproduces these claims; it is ch01-noisy-bit.ipynb in the directory bernoulli-py/notebooks/ of the companion package. It estimates \(\fprate \) from a simulated binary symmetric channel across sample sizes spanning several orders of magnitude, exhibiting the \(1/\sqrt {n_0}\) concentration of \(\hat {\fprate }\) about the true rate; and it measures the empirical coverage of the \(95\%\) Wald interval, showing it approach nominal at moderate \(n_0\) and fall below nominal as \(n_0\) shrinks or \(\fprate \) approaches a boundary, the under-coverage noted above.

1.5 Why “Atom”?

The noisy bit is atomic in two senses. First, it has no internal Bernoulli structure to decompose: the input is one Boolean value, the output is one Boolean value, and the entire behavior is two numbers, \(\fprate \) and \(\fnrate \). A smaller part would require a sub-Boolean input or output, and there is nothing below a Boolean. The atom is where internal accounting stops and external accounting begins, and every larger object in the book, chained channels, noisy logic gates, randomized algorithms, approximate sets, general algebraic types, is assembled from noisy bits by a specific composition rule. Second, the noisy bit sits at the minimum of the model-order ladder that Section 2.4 defines, the number of distinct error regimes an input domain splits into. A general noisy bit has model order two, the blocks \(\{\top \}\) and \(\{\bot \}\) carrying independent rates; the symmetric case collapses to order one. No noisy system over a Boolean domain has a smaller non-trivial channel matrix.

The claim is a working hypothesis until discharged, and chapters 2 through 4 do so in three directions. Chapter 2 gives the noisy bit a \(2 \times 2\) row-stochastic matrix \(Q\), places the symmetric, asymmetric, and Z-channel cases on the model-order ladder, and shows the atom closed under serial composition (the matrix product of row-stochastic matrices is row-stochastic). Chapter 3 asks what AND, OR, and NOT do to noisy bits. Gate by gate the answer is closed-form, but it is not again a noisy bit: a gate over two distinct latent bits has a single rate on one side and up to three on the other, so a pair of rates does not propagate through a Boolean tree. What does propagate exactly is the error at each latent assignment of the inputs; a single pair survives only as a prior average or as a worst-case bound (Section 3.3), and corner products such as the composition theorem (Section 3.4) are exact only in their stated regimes. One round of the Miller-Rabin primality test is a noisy bit with \(\fnrate = 0\) and \(\fprate \le 1/4\), and because every round reads the same latent value the \(k\)-round bound \(\fprate ^{(k)} \le (1/4)^k\) is the all-negative product rule rather than a separate analysis (Section 3.5). Chapter 4 names the two independence assumptions used silently until then, element-wise independence (Section 4.2) and conditional independence of per-block rates (Section 4.3); proves the Kronecker factorization they purchase (Section 4.4), which is what makes the per-element description sufficient; and writes the formal definition \(\Bernoulli {\BoolType } = (X, Y, Q)\) (Section 4.5), of which every atom in chapters 1 through 3 is an instance.

Beyond Part I the atom is held fixed and its surroundings vary. Part II keeps the Boolean codomain and arranges many noisy bits into a structure: a set \(S\) over a universe induces a membership query per element, an approximate set makes each query a noisy bit (the Bloom filter being the canonical one-sided instance, \(\fnrate = 0\) with tunable \(\fprate \)), and the development is the algebra of how set operations propagate error, how classification measures quantify quality, and how much space a target accuracy demands. Part III replaces the Boolean codomain with an arbitrary algebraic type, lifting the atom to approximate maps, relations, and structured values, where the two-element model order ceases to be a ceiling. The atom is the unit that gets composed; everything that follows is composition, and an honest account of what composition preserves and what it does not.

1.6 Notes and Lineage

BF Lineage. Every ingredient of the noisy bit is classical. The channel reading, a conditional distribution of output given input, and the binary symmetric channel are Shannon’s [Sha48]. The \(2 \times 2\) table of sensitivity and specificity comes from diagnostic testing, where Bross [Bro54] studied how misclassification distorts it. The same asymmetric channel applied deliberately to a survey respondent’s true indicator is randomized response [War65], deployed at scale in RAPPOR [EPK14]; there the channel is known and inverted to estimate a prevalence, the reverse of the error-accounting use made of it here. Precision and recall come from information retrieval [MRS08, Ch. 8]. The binomial estimators and intervals of Section 1.4 are standard and are developed, with their sources, in chapter 7; Miller-Rabin is credited in the notes to chapter 3. The framing of these objects as the atom of a framework of approximate types is that of the source manuscripts [Tow26b; Tow26k].

BF What this chapter adds. A presentation: the biased coin, the noisy line, and the classifier verdict are treated as three faces of one atom with two parameters, and the distinction between the intrinsic pair \((\fprate , \fnrate )\) and the workload-dependent pair (precision, recall) is fixed as the book’s convention. No result in the chapter is new.

BF Left out. ROC analysis and threshold selection, cost-sensitive decision rules, and the inversion of a known channel to estimate prevalence (the randomized-response problem) are mentioned only in passing.

Chapter 2
Binary Channels and Model Order

2.1 The Binary Symmetric Channel

The three framings of Section 1.1 share a common structure, and the binary symmetric channel is that structure stripped to its essentials: a single Boolean input, a single Boolean output, and one crossover probability that the output disagrees with the input.

The defining property is that the crossover is the same for both inputs: \(\Pr \{\phi (\top ) \neq \top \} = \Pr \{\phi (\bot ) \neq \bot \} = \fprate \), so a single parameter plays the role of both the false-positive and the false-negative rate, \(\fprate = \fnrate \). The symmetry is a genuine constraint, and most real systems lack it: a spam filter flags legitimate mail far more often than it leaks spam. The BSC is the clean starting point precisely because it isolates bit-flip noise from any question of which direction is noisier; the asymmetric generalization is Section 2.2.

The channel matrix

We record a binary channel as a \(2 \times 2\) row-stochastic matrix \(Q\), the channel matrix (Figure 2.1(a) draws the BSC case). The convention, fixed for the rest of the book: rows are indexed by the input block, row 1 for \(B_1 = \{\top \}\) (positives) and row 2 for \(B_2 = \{\bot \}\) (negatives); columns by the output label, \(\top \) then \(\bot \); and the \((i,j)\) entry is the probability of output \(j\) given an input in block \(B_i\). The four entries name the confusion-table rates, so the general order-2 channel and the BSC special case are \[ Q \;=\; \begin {pmatrix} \tprate & \fnrate \\ \fprate & \tnrate \end {pmatrix} \;=\; \begin {pmatrix} 1 - \fnrate & \fnrate \\ \fprate & 1 - \fprate \end {pmatrix}, \qquad Q_{\mathrm {BSC}} \;=\; \begin {pmatrix} 1 - \fprate & \fprate \\ \fprate & 1 - \fprate \end {pmatrix}. \] The BSC matrix is circulant: its two rows are permutations of each other, the matrix expression of symmetry. At \(\fprate = 0.05\), for instance, a positive input reads back correctly with probability \(0.95\) and flips with probability \(0.05\), and likewise for a negative input; under a uniform input prior the marginal error rate is exactly \(\fprate \), independent of the prior because the channel is symmetric.

Capacity

The BSC’s capacity, the maximum mutual information \(I(X; Y)\) over input priors, is \[ C \;=\; 1 - H_2(\fprate ), \qquad H_2(\fprate ) = -\fprate \log _2 \fprate - (1 - \fprate )\log _2(1 - \fprate ), \] attained at the uniform prior \(p = 1/2\), with \(H_2\) the binary entropy function; both the channel and the formula are Shannon’s [Sha48]. The derivation is short: \(H(Y \mid X) = H_2(\fprate )\) for every prior, while \(H(Y)\) is maximized at \(p = 1/2\), so \(C = \max _p H(Y) - H_2(\fprate ) = 1 - H_2(\fprate )\). Capacity is one bit per use at \(\fprate = 0\) (a perfect channel) and zero at \(\fprate = 1/2\), where the output is independent of the input. The \(\fprate = 1/2\) point is the zero-information case met in Section 1.2; for the BSC it coincides with the symmetric crossover, whereas for an asymmetric channel the zero-information locus is \(\fprate + \fnrate = 1\).

The three faces in matrix form

The framings of Section 1.1 are the same matrix \(Q_{\mathrm {BSC}}\) with different labels. The biased coin is a coin with one flip probability \(\fprate \) for both faces; the noisy line is Shannon’s original BSC [Sha48], a bit flipped independently with probability \(\fprate \); the classifier with equal false-positive and false-negative rates is a BSC, though real classifiers rarely satisfy \(\fprate = \fnrate \) and are better modeled by the asymmetric channel of Section 2.2. Composing a BSC with itself makes matters worse, not better: two crossover-\(\fprate \) channels in series give crossover \(2\fprate (1 - \fprate ) > \fprate \) for \(0 < \fprate < 1/2\), derived in Section 2.3.

Model order of the BSC

The source literature uses two phrases for the BSC, “first-order” and “the symmetric special case of the order-2 model,” and they refer to different things. Model order, formalized in Section 2.4, counts the blocks in the partition of the input domain that groups inputs sharing a common error rate. For the BSC both inputs share the crossover \(\fprate \), so they form a single block and the BSC has model order 1; the asymmetric channel, with \(\fprate \neq \fnrate \), keeps them in separate blocks at model order 2. Calling the BSC “the symmetric special case of the order-2 model” refers not to its order but to the family it is embedded in: one may always write the BSC with the two-block discrete partition and impose \(\fprate = \fnrate \), the working frame for chapters 2 through 4. The BSC is genuinely model order 1, studied as a constrained member of the order-2 family; Section 2.4 separates model order from parameter count and proves that no binary channel exceeds order 2.

Throughout, \(\fprate \) is the false-positive rate, \(\fnrate \) the false-negative rate, with \(\tprate = 1 - \fnrate \) and \(\tnrate = 1 - \fprate \) the complementary rates, and for the BSC the single crossover \(\fprate = \fnrate \). The source papers sometimes pair (FPR, TPR) rather than (FPR, FNR); since \(\mathrm {TPR} = 1 - \mathrm {FNR}\) the choice is cosmetic, and this book uses \((\fprate , \fnrate )\).

2.2 The Asymmetric Binary Channel

The BSC’s equal error rates are rarely satisfied in practice: a spam filter, a Bloom filter, and a medical screening test all treat their two error directions differently. Allowing the false-positive rate \(\fprate \) and the false-negative rate \(\fnrate \) to vary independently gives the general binary channel, the atom of two-sided error.

Setting \(\fnrate = \fprate \) recovers the BSC of Definition 2.1.1. Setting \(\fnrate = 0\) makes row 1 equal to \((1, 0)\), so a positive input is always reported correctly: this is the Z-channel. Figure 2.1 draws the three cases. The asymmetric channel is the standard binary channel of information theory. Read as a model of how a recorded indicator departs from the true one, it is also the misclassification model of \(2 \times 2\) tables and, when the noise is introduced deliberately, Warner’s randomized response (see the Notes and Lineage at the end of this chapter).

The Z-channel makes false negatives impossible while admitting false positives at rate \(\fprate \) (one-sided error), so an output of \(\bot \) is fully reliable and an output of \(\top \) is uncertain.¹ This reliable-no, uncertain-yes profile is exactly the Bloom filter: it answers set membership with no false negatives (\(x \in S\) always returns \(\top \)) and a tunable false-positive rate (\(x \notin S\) may still return \(\top \)), so a Bloom filter query is a Z-channel, \(\fnrate = 0\) with \(\fprate \) set by its bit-width and hash count (conditionally on the realized filter; Chapter 5 treats the randomness of the rate itself). The doubly-biased coin, the screening test (small \(\fnrate \), larger \(\fprate \)) and its confirmatory counterpart (the reverse), and any system tuned by a decision threshold are all asymmetric channels; as the threshold sweeps, \((\fprate , \fnrate )\) traces a curve in \([0,1]^2\), each point a distinct channel, which is the ROC curve re-axed.

For one instance, take \(\fprate = 0.05\), \(\fnrate = 0.10\): \[ Q \;=\; \begin {pmatrix} 0.90 & 0.10 \\ 0.05 & 0.95 \end {pmatrix}, \] with false negatives twice as likely as false positives. Under a uniform input prior its accuracy is the diagonal average \(\tfrac {1}{2}(0.90 + 0.95) = 0.925\). A BSC at \(\fprate = \fnrate = 0.075\) achieves the same accuracy with both off-diagonal entries equal, so accuracy alone does not see the asymmetry; the distinction matters once the prior is non-uniform or the two errors carry different costs, which is the subject of Chapter 7.

The general order-2 case

The asymmetric channel is the general model-order-2 binary channel: the input domain partitions into \(B_1 = \{\top \}\) and \(B_2 = \{\bot \}\), each block carrying its own rate, \(\fnrate \) for \(B_1\) and \(\fprate \) for \(B_2\). The BSC imposes \(\fnrate = \fprate \), which merges the two blocks into one and drops the model order to 1 (Section 2.4); the Z-channel keeps the blocks distinct (\(\fnrate = 0 \neq \fprate \)) and so remains order 2 with a single free parameter, the first sign that model order and parameter count are separate quantities. Section 2.4 makes the distinction precise and proves the Boolean ceiling: with only two input elements, no binary channel exceeds model order 2.

2.3 Composition of Channels

Many systems chain noisy steps: an amplifier feeds another, a filter feeds a ranker. If an element passes through \(Q_1\) and its output enters \(Q_2\), the end-to-end channel is the matrix product.

The sum is the law of total probability over the intermediate value \(k\): \((Q_1)_{ik}\) carries block \(B_i\) to intermediate output \(k\), and \((Q_2)_{kj}\) carries \(k\) to final output \(j\). Because the matrices are row-stochastic and act on row vectors of input probabilities, the stage applied first is the left factor: an input distribution \(p\) leaves the chain as \(p\,Q_1 Q_2\). This is the two-step Chapman-Kolmogorov relation of a Markov chain, and the cascade of channels of information theory [CT06].

Composing two BSCs

Two BSCs in series produce a third. Multiplying circulant matrices preserves the circulant form, and the off-diagonal entry of \(Q_1 Q_2\) is \begin {equation} \label {eq:bsc-composition} \fprate _{\mathrm {total}} \;=\; \fprate _1(1 - \fprate _2) + (1 - \fprate _1)\fprate _2 \;=\; \fprate _1 + \fprate _2 - 2\fprate _1\fprate _2. \end {equation} The composed error is the exclusive-or of the two individual flips: the input is corrupted exactly when one channel flips and the other does not (if both flip, the errors cancel). For \(0 < \fprate _1, \fprate _2 < 1/2\) this exceeds both operands, so two imperfect symmetric channels in series are noisier than either alone; at \(\fprate _1 = 0.05\), \(\fprate _2 = 0.10\) the composed crossover is \(0.14\). Chaining copies of one BSC drives the crossover toward the stationary \(1/2\), where the channel carries no information.

Composing asymmetric channels

The general rule is the same matrix product. For \(Q_1, Q_2\) with rates \((\fprate _i, \fnrate _i)\), the \((1,2)\) and \((2,1)\) entries of \(Q_1 Q_2\) give \begin {equation} \label {eq:asym-composition} \fnrate _{\mathrm {total}} = (1 - \fnrate _1)\fnrate _2 + \fnrate _1 (1 - \fprate _2), \qquad \fprate _{\mathrm {total}} = \fprate _1 (1 - \fnrate _2) + (1 - \fprate _1)\fprate _2 . \end {equation} Each term follows one path through the intermediate value: a true \(\top \) ends as \(\bot \) either by surviving stage 1 and being dropped by stage 2, or by being dropped at stage 1 and then not being restored to \(\top \) by stage 2 (which treats the intermediate \(\bot \) as a negative). At \((\fprate _1, \fnrate _1) = (0.05, 0.10)\) and \((\fprate _2, \fnrate _2) = (0.02, 0.20)\) the chain has \(\fnrate _{\mathrm {total}} = 0.278\) and \(\fprate _{\mathrm {total}} = 0.059\). Composition of asymmetric channels is not generally symmetric: the BSC is the special case in which symmetry is preserved because both factors are symmetric, forcing \(\fprate _{\mathrm {total}} = \fnrate _{\mathrm {total}}\).

Serial versus parallel

Serial composition must not be confused with the parallel query structure of Section 4.4 (the Kronecker product). Serial composition feeds one element’s output into the next channel and yields a \(2 \times 2\) matrix, the ordinary product \(Q_1 Q_2\). Parallel querying applies \(m\) independent channels to \(m\) distinct elements and yields a \(2^m \times 2^m\) joint matrix, the Kronecker product \(Q_1 \otimes \cdots \otimes Q_m\), which Theorem 4.4.1 shows is forced by the independence axioms. The two answer different questions, end-to-end error of a pipeline versus joint distribution of simultaneous queries, and combine when \(k\) multi-stage pipelines run in parallel: the Kronecker product across pipelines, the ordinary product within each. The serial product needs only the law of total probability; the Kronecker product needs the axioms (Corollary 4.4.1.1).

Algebraic properties

Serial composition is associative, since matrix multiplication is, so a chain of channels groups unambiguously. It is not commutative in general: \(Q_1 Q_2 \neq Q_2 Q_1\) unless the factors share a symmetry. Two BSCs do commute, because (2.1) is symmetric in \(\fprate _1\) and \(\fprate _2\); a BSC and an asymmetric channel generally do not, as a direct computation of \(Q_1 Q_2\) and \(Q_2 Q_1\) shows, so the order of stages in a mixed pipeline matters.

For one-sided stages of the same orientation the product simplifies. With \(\fnrate _1 = \fnrate _2 = 0\), (2.2) gives \(\fnrate _{\mathrm {total}} = 0\) and \(\fprate _{\mathrm {total}} = 1 - (1 - \fprate _1)(1 - \fprate _2)\), the form \(\eta _{\mathrm {total}} = 1 - \prod _{i}(1 - \eta _i)\) that Section 3.4 states for chains in which any single stage error is fatal.

2.4 Model Order

The BSC carries one crossover parameter and the asymmetric channel two, one per input. Whether a binary channel could carry more is a question about the input domain, answered by model order.

Partitions and blocks

A partition of a set \(S\) is a collection of non-empty, pairwise disjoint blocks \(B_1, \ldots , B_n\) whose union is \(S\). In the channel model the relevant partition is of the input domain: elements of one block share a per-block error rate, and Axiom 2 (Section 4.3) treats the per-block rates \(\{\alpha _1, \ldots , \alpha _n\}\) as the channel’s free parameters.

The Boolean ceiling

The Boolean domain \(D = \{\top , \bot \}\) admits exactly two partitions: the trivial one, \(\{\{\top , \bot \}\}\), with both inputs in a single block sharing one rate; and the discrete one, \(\{\{\top \}, \{\bot \}\}\), with each input carrying its own rate.

The BSC realizes order 1: both inputs share the crossover \(\fprate \), so the trivial partition is compatible and is the coarsest such, with channel matrix \(Q_{\mathrm {BSC}}\) from Definition 2.1.1. The asymmetric channel of Definition 2.2.1 realizes order 2: with \(\fprate \neq \fnrate \) the two inputs cannot share a block. The ceiling is tight (order 2 is attained) but not forced (the BSC sits at order 1). Serial composition stays within it: composing binary channels (Section 2.3) yields a binary channel, still of model order at most 2. The ceiling is a statement about a two-element input domain, and it is the reason Boolean gates behave differently: a gate over two distinct latent bits is a channel from the four-element domain of latent pairs, and its output can carry up to four distinct error regimes (Section 3.3). Composition of gates raises model order; serial composition of binary channels does not.

Model order versus parameter count

Model order is not the same as the number of free parameters, and conflating them is a common error. Model order counts partition blocks; parameter count counts the real numbers needed to specify the channel. For a single Boolean query an \(n\)-th order channel has \(n\) free parameters, one per block, so the two agree, but they part company in two ways. First, a constraint can reduce the parameter count below the order: the Z-channel of Definition 2.2.2 has model order 2 (its blocks carry the distinct rates \(0\) and \(\fprate \)) yet only one free parameter, since \(\fnrate = 0\) is fixed. Second, in the multi-query setting the parameter count grows with the number of queries while the model order does not: a general order-\(n\) channel on \(m\) joint inputs carries \(n(2^m - 1)\) parameters in the per-source-block convention, which the independence axioms collapse to \(nm\) by Kronecker factorization (Corollary 4.4.1.2), all while the model order stays \(n\). Model order is likewise distinct from \(k\)-fold composition, the number of times an approximate identity is iterated, which changes rates but not the partition.

The BSC and the perfect channel merge their two blocks (equal rates) and sit at order 1; the Z-channel and the asymmetric channel keep distinct rates and sit at order 2. The Z-channel row is the clean separation of the two counts: order 2, one parameter.

Axiom 2 and the n-ary ceiling

The partition-block structure is exactly what Axiom 2 ( Axiom  2) acts on: it asserts that the per-block rates are conditionally independent given the objective set, and the model order is the dimension of that independence claim. In the order-2 Boolean case this is the independence of \(\fprate \) and \(\fnrate \). The Boolean ceiling bounds the claim at two factors, which is what keeps the framework tractable: every Boolean channel is at most two parameters per query, and the Kronecker factorization reduces the \(m\)-query count to \(2m\). The machinery is not special to Booleans. For an input domain of cardinality \(k\) the same partition argument gives a ceiling of \(k\), the subject of Chapter 9; Part I develops the order-2 Boolean case, from which the general theory is the natural lift.

2.5 Notes and Lineage

BF Lineage. The channel matrix, the binary symmetric channel, and its capacity \(1 - H_2(\fprate )\) are Shannon’s [Sha48]; the asymmetric binary channel and the Z-channel are standard textbook objects [CT06, Ch. 7]. Serial composition as a product of row-stochastic matrices is the law of total probability, the cascade of channels and the two-step Chapman-Kolmogorov relation. The same \(2 \times 2\) object arose independently in biostatistics, where Bross [Bro54] analyzed misclassification in \(2 \times 2\) tables, and in survey design, where Warner [War65] passed each respondent’s true indicator through a known asymmetric channel and recovered the population rate by inverting it. Model order, the number of blocks in the coarsest partition of the input domain whose elements share an error probability, is the organizing vocabulary of the source manuscripts [Tow26b; Tow26k]; the Boolean ceiling (Theorem 2.4.1) is the elementary fact that a two-element set has at most two blocks.

BF What this chapter adds. A single row convention for \(Q\) used throughout the book, the explicit asymmetric composition formulas (2.2) with the order of factors fixed by that convention, and the separation of model order from parameter count: the Z-channel has order two and one free parameter, and the \(m\)-query count is governed by the axioms of chapter 4, not by the order. The forward link that matters most is negative: serial composition never leaves the order-two ceiling, but Boolean gates over distinct latent bits do (Section 3.3).

BF Left out. Channel coding, capacity-achieving codes, and channels with memory are not treated; Cover and Thomas [CT06] is the reference. The estimation-theoretic use of \(Q\) (inverting a known channel to recover prevalence, as randomized response and readout-error mitigation do) returns only in chapters 7 and 19.

Chapter 3
Boolean Algebra Under Noise

3.1 AND Under Noise

A \(\Bernoulli {\BoolType }\) instance is a noisy bit with rates \((\fprate , \fnrate )\) (Definition 4.5.1). The first question of the chapter is what a Boolean operation does to two of them. AND is the natural start: it is the operation behind a Bloom filter’s \(k\)-hash query and Miller-Rabin’s \(k\)-round test, and OR and NOT follow from it (Section 3.2).

Let \(\tilde {b}_1, \tilde {b}_2\) be independent instances with rates \((\fprate _i, \fnrate _i)\) observing two latent (true) bits \(b_1, b_2\), and apply AND to their observed outputs; the true value is the AND of the latent bits. Two independence assumptions are in play, and the chapter’s purpose is to invoke them deliberately. Axiom 1 ( Axiom  1) makes the error events of the two instances independent given the latent bits, licensing every product below; Axiom 2 ( Axiom  2) makes each instance’s rates stable constants rather than a drifting process. Chapter 4 proves both and shows they are the content of the Kronecker factorization (Section 4.2, Section 4.3).

The AND of the observed outputs is \(\top \) exactly when both report \(\top \), which by Axiom 1 happens with the following probabilities across the four latent pairs:

Output rates

The true AND is \(\top \) only in the first row, so a false negative (true \(\top \), observed \(\bot \)) is the complement of that row’s success: \begin {equation} \label {eq:and-fnr} \fnrate _\wedge \;=\; 1 - (1 - \fnrate _1)(1 - \fnrate _2) \;=\; \fnrate _1 + \fnrate _2 - \fnrate _1 \fnrate _2. \end {equation} This depends only on the input rates, not on the input prior: the true AND is \(\top \) at exactly one latent pair, \((\top , \top )\), so there is exactly one false-negative rate, and the AND succeeds there only if both instances do.

The false-positive rate is different. The true AND is \(\bot \) at three latent pairs, and the observed AND is \(\top \) there with the three conditional probabilities \begin {equation} \label {eq:and-negative-rows} (\top ,\bot ):\ (1 - \fnrate _1)\fprate _2, \qquad (\bot ,\top ):\ \fprate _1(1 - \fnrate _2), \qquad (\bot ,\bot ):\ \fprate _1 \fprate _2 . \end {equation} These are generally three different numbers. Viewed as a channel from the latent pair to the reported bit, the AND output has one positive row and three negative rows, so in the sense of Definition 2.4.1, applied to the four-element domain of latent pairs, it can have model order up to four. No single false-positive rate describes it: the AND of two \(\Bernoulli {\BoolType }\) instances on distinct latent bits is not again a two-rate noisy bit. Section 3.3 develops the consequence for formulas, including a counterexample (Example 3.1) in which reusing a single rate halves the true error.

A single number arises only by averaging. If the two latent bits are independent with common prevalence \(\pi = \Pr [b_i = \top ]\), the probability that a randomly drawn negative pair is reported \(\top \) is the prior-weighted mixture of (3.2), \begin {equation} \label {eq:and-fpr-general} \fprate _\wedge \;=\; \frac {\pi (1 - \pi )\bigl [(1 - \fnrate _1)\fprate _2 + \fprate _1(1 - \fnrate _2)\bigr ] + (1 - \pi )^2 \fprate _1 \fprate _2} {1 - \pi ^2}. \end {equation} This is a prior-averaged rate: an exact statement about the gate under the stated input distribution, and the right number for a workload drawn from it, but not the conditional rate at any particular input, and not a parameter a downstream gate can reuse except in the narrow case of Remark 3.3.2. The two extremes are instructive. When both inputs are negative (the all-negative regime, \(\pi \to 0\) or both inputs known negative) the boundary rows drop out and \[ \fprate _\wedge \;=\; \fprate _1 \fprate _2, \] the product, which drives the false-positive rate down: two noisy bits both have to err for the conjunction to err. When one input is positive, the relevant row contributes \(\approx \fprate \) (only one instance need err), so the mixture lies between \(\fprate _1 \fprate _2\) and \(\max (\fprate _1, \fprate _2)\). The all-negative product regime is the one Miller-Rabin operates in: its \(k\) rounds all test the same composite, so every round observes the same latent value \(\bot \) and the conjunction’s false-positive rate is the clean product \(\prod _i \fprate _i\) (Section 3.5).

AND therefore maps two \(\Bernoulli {\BoolType }\) instances to a channel with a single false-negative rate (3.1) and a false-positive rate that depends on which negative latent pair occurred: the product at \((\bot , \bot )\), the larger boundary values at \((\top , \bot )\) and \((\bot , \top )\). For two independent fair coins read through channels with rates \((0.10, 0.05)\) and \((0.08, 0.12)\), reporting whether both landed heads gives \(\fnrate _\wedge = 0.164\) by (3.1). The three negative rows are \(0.076\), \(0.088\), and \(0.008\), and at \(\pi = 0.5\) their average (3.3) is \(\fprate _\wedge \approx 0.057\), a number that matches none of them. If instead both channels read the same coin, only the latent pairs \((\top , \top )\) and \((\bot , \bot )\) occur and the false-positive rate is exactly the product \(0.008\): this shared-truth reading is the Miller-Rabin regime, and it is the reading under which the companion package’s BernoulliBool conjunction returns \((\fprate _1 \fprate _2, 1 - (1 - \fnrate _1)(1 - \fnrate _2))\). AND trades false-negative accuracy for false-positive suppression, but by an amount that depends on the input.

3.2 OR and NOT

NOT is immediate and OR follows from it by De Morgan, completing the Boolean basis.

NOT

Complementing the output of a \(\Bernoulli {\BoolType }\) instance turns each false negative into a false positive and conversely: a \(\bot \) reported on a true \(\top \) becomes a \(\top \) reported on the complemented true \(\bot \). So NOT swaps the rate pair, \((\fprate , \fnrate ) \mapsto (\fnrate , \fprate )\), which on the channel matrix is the row exchange \begin {equation} \label {eq:not-Q} Q_{\neg } \;=\; \begin {pmatrix} 1 - \fprate & \fprate \\ \fnrate & 1 - \fnrate \end {pmatrix}. \end {equation} NOT is an involution on \(\Bernoulli {\BoolType }\): applying it twice restores the rates, the noisy-bit form of \(\neg \neg b = b\) (at the level of the channel matrix, not of any realized output). Unlike AND and OR, NOT acts on a single latent bit, so it maps a two-rate instance to a two-rate instance exactly.

OR

By De Morgan, \(\tilde {b}_1 \vee \tilde {b}_2 = \neg (\neg \tilde {b}_1 \wedge \neg \tilde {b}_2)\). Negating the inputs swaps their rates, the AND of the negated instances has false-negative rate \(\fprate _1 + \fprate _2 - \fprate _1 \fprate _2\) by (3.1) with rates swapped, and the outer NOT swaps that into the OR’s false-positive rate. The truth table confirms it directly: the observed OR is \(\bot \) only when both instances report \(\bot \), so a false positive (true \(\bot \bot \), observed \(\top \)) is \begin {equation} \label {eq:or-fpr-direct} \fprate _\vee \;=\; 1 - (1 - \fprate _1)(1 - \fprate _2) \;=\; \fprate _1 + \fprate _2 - \fprate _1 \fprate _2, \end {equation} the inclusion-exclusion probability that at least one instance false-positives. There is one negative latent pair, so this rate is single-valued and prior-free. The OR’s false-negative rate, dually, takes three values over the three positive latent pairs, \(\fnrate _1 \fnrate _2\) at \((\top , \top )\), \(\fnrate _1(1 - \fprate _2)\) at \((\top , \bot )\), and \((1 - \fprate _1)\fnrate _2\) at \((\bot , \top )\); it reduces to the product \(\fnrate _1 \fnrate _2\) only in the all-positive regime.

The duality is exact: AND has a single false-negative rate (3.1) and a latent-dependent false-positive rate (3.2), while OR has a single false-positive rate (3.5) and a latent-dependent false-negative rate. AND suppresses false positives and inflates false negatives; OR does the reverse; NOT exchanges the two regimes. In the symmetric case \(\fprate _i = \fnrate _i = \fprate \), the single-valued sides inflate by the same \(2\fprate - \fprate ^2\), and De Morgan’s duality is manifest.

The one-sided closure under OR is the reason the union of two Bloom filters is again a Bloom-type structure with no false negatives (Chapter 6); the failure of closure under AND is the reason their intersection is not, its false-positive rate depending on whether an element lies in one operand or neither. Repeated independent tests of a single latent bit are the other exception: only the latent pairs \((\top , \top )\) and \((\bot , \bot )\) can occur, and the conjunction is a two-rate instance with rates \((\fprate _1 \fprate _2, 1 - (1 - \fnrate _1)(1 - \fnrate _2))\), the structure of Miller-Rabin. For general instances the gate formulas remain exact, but only per latent pair: what a whole formula inherits is the subject of the next section, where a two-rate summary is shown to mislead and two sound replacements are given.

3.3 Boolean Formulas Under Noise

The gate formulas of Section 3.1 and Section 3.2 describe one gate whose inputs observe distinct latent bits. A formula feeds gate outputs into further gates, and the natural hope, that each gate’s output can be summarized by a pair \((\fprate , \fnrate )\) and the summary passed upward, fails. This section states what is exact, shows by example how the two-rate summary fails, and gives the two summaries that are sound.

Exact propagation, one latent assignment at a time

The read-once rules are the signal-probability calculus that Parker and McCluskey gave for combinational networks [PM75]; probabilistic transfer matrices are its matrix form for general circuits, including reconvergent fan-out [Kri+08]. A variable that occurs more than once is still one observation feeding several places, so its occurrences are perfectly correlated; conditioning on the observed values of the repeated variables (a Shannon expansion) restores the read-once case at a cost of \(2^r\) passes for \(r\) repeated variables. The companion module bernoulli.formula implements the proposition: its error_probability and error_profile evaluate \(e_f(w)\) at one or all latent assignments.

Why a rate pair does not compose

Viewed as a channel from latent assignments to the reported bit, a formula’s output has one row per latent assignment, and its model order in the sense of Definition 2.4.1, applied to the domain \(\{0,1\}^n\), is the number of distinct values \(e_f(w)\) takes. A two-rate description is exact only when that number is at most one per truth class. The AND of two distinct latent bits already has three negative rows (3.2): composition raises the model order. Carrying a single number per gate upward therefore discards which latent assignment occurred, and the error compounds.

Example 3.1 (A two-rate summary halves the error) Take \(f = (b_0 \wedge b_1) \vee b_2\) with every channel at \(\fprate = 0.02\) and \(\fnrate = 0.10\), and the latent assignment \(w = 010\) (\(b_0\) false, \(b_1\) true, \(b_2\) false), at which \(f(w) = \bot \). The AND observes the pair \((\bot , \top )\) and reports \(\top \) with probability \(\fprate (1 - \fnrate ) = 0.018\); the OR reports \(\bot \) only if both of its inputs do, so \[ e_f(010) \;=\; 1 - (1 - 0.018)(1 - 0.02) \;=\; 1 - (0.982)(0.98) \;=\; 0.03764. \] A two-rate summary that carries the AND’s all-negative rate \(\fprate ^2 = 0.0004\) upward gives \(1 - (1 - 0.0004)(0.98) = 0.020392\), about half the true value; it is the exact error at \(w = 000\) only. Exhaustive enumeration of the eight latent assignments gives two distinct false-positive values (\(0.020392\) at \(000\); \(0.03764\) at \(010\) and \(100\)) and four distinct false-negative values (\(0.09996\) at \(001\); \(0.0982\) at \(011\) and \(101\); \(0.1862\) at \(110\); \(0.019\) at \(111\)): six error regimes, model order six on the latent domain, for a formula built from three identical two-rate atoms (Figure 3.1).

Two sound summaries

When a single pair is wanted anyway, two constructions are sound.

On Example 3.1 the recursion gives \((0.02, 0.19)\) for the AND and \((1 - 0.98^2, \max (0.19, 0.10)) = (0.0396, 0.19)\) for the formula, which bounds the exact worst values \(0.03764\) and \(0.1862\). The exact worst pair, the maximum of \(e_f\) over each truth class, is sharper but needs the enumeration of Proposition 3.3.1 (bernoulli.formula.worst_case_rates); the recursive bound costs one pass. Neither is the exact rate at a given input, and the gap between the bound and the corner product is real: in the all-negative regime the AND’s false-positive rate is \(\fprate _g \fprate _h\) (Section 3.4), far below \(\max (\bar {E}_g, \bar {E}_h)\), because there the latent assignment is known. That regime, and the one-sided chains in which a single error is fatal, are where corner products are exact; the next section states them as the composition theorem.

3.4 The Composition Theorem

The AND’s false-negative rate (3.1) and the OR’s false-positive rate (3.5) are both instances of one pattern: the probability that a chain produces no error factors as a product. This section states it as a theorem and isolates the regimes where it, and its dual, are exact. The theorem is the elementary product rule for a series system of independent components; its use here is to say precisely which corner of Section 3.3 it describes.

The inductive step uses “if and only if both,” which holds only when any single error is fatal. If a later stage can accidentally undo an earlier error, the formula overcounts; this is the BSC caveat below.

Which rate each direction governs

The theorem applies to the error direction in which a single stage error propagates. For the conjunction of \(k\) instances (report \(\top \) iff all do), a true \(\top \), which for an AND means the single latent assignment with every input \(\top \), survives only if every stage reports \(\top \), so correctness multiplies and the false-negative rate is \[ \fnrate _{\wedge ,k} = 1 - \prod _{i=1}^{k}(1 - \fnrate _i), \] the theorem with \(\eta _i = \fnrate _i\). Dually, the disjunction’s false-positive rate is \(\fprate _{\vee ,k} = 1 - \prod _i (1 - \fprate _i)\).

The opposite direction is the dual: the conjunction false-positives on an all-negative input only if every stage false-positives, so errors multiply and \begin {equation} \label {eq:composition-theorem} \fprate _{\wedge ,k} = \prod _{i=1}^{k} \fprate _i \end {equation} in the all-negative regime, that is, at the one latent assignment with every input \(\bot \). This is the mechanism behind Miller-Rabin: its \(k\) rounds all test the same composite, the only latent value is \(\bot \), and the false-positive rate is the product \(\prod _i \fprate _i \le (1/4)^k\) (Section 3.5). The two formulas are not interchangeable: the conjunction’s false-negative rate is \(1 - \prod (1 - \fnrate _i)\), but its false-positive rate in the all-negative regime is the product \(\prod \fprate _i\), not \(1 - \prod (1 - \fprate _i)\). At any other negative latent assignment the false-positive rate is larger, a product in which each \(\top \) input contributes a factor \(1 - \fnrate _i\) rather than \(\fprate _i\) (3.2); carrying the all-negative product into a context where the latent assignment is not all-negative is the error of Example 3.1.

The BSC caveat

The theorem computes the probability that at least one stage errs. For the identity-with-error and one-sided cases this equals the end-to-end error rate, but the BSC violates the “single error is fatal” premise: two flips cancel. Two BSCs of crossover \(\fprate \) in series have end-to-end crossover \(2\fprate - 2\fprate ^2\) (the odd-flip probability, Equation (2.1)), whereas the theorem’s \(1 - (1 - \fprate )^2 = 2\fprate - \fprate ^2\) counts the both-flip event as an error even though it returns the correct value; the exact rate is the parity of the flips (Remark 3.3.1). The two agree only when cancellation is impossible: one-sided channels (\(\fnrate = 0\) or \(\fprate = 0\)), and the AND and OR gates in their respective monotone directions. For the BSC, the matrix product of Section 2.3 is the correct tool, not this theorem. The two formulas answer different questions about different models.

The Miller-Rabin regime, worked

Take \(k\) one-sided instances (\(\fnrate _i = 0\)), the Bloom-filter and Miller-Rabin setting, queried in the all-negative regime, with false-positive rates \(\fprate _i\). The conjunction has \(\fnrate _{\wedge ,k} = 0\) (no stage ever suppresses a true \(\top \)), and its false-positive rate is the product (3.7). For \(\fprate = (0.20, 0.15, 0.10, 0.25)\), \[ \fprate _{\wedge ,4} = 0.20 \cdot 0.15 \cdot 0.10 \cdot 0.25 = 7.5 \times 10^{-4}, \] far below any single stage: every stage must independently false-positive for the conjunction to. With all four at \(\fprate = 1/4\) the product is \((1/4)^4 \approx 3.9 \times 10^{-3}\), the Miller-Rabin bound at \(k = 4\). The conjunction of one-sided tests drives the false-positive rate down geometrically, which is exactly the mechanism Miller-Rabin exploits.

The parallel counterpart to this serial result is the Kronecker factorization (Theorem 4.4.1): correctness multiplies along a chain, while the joint confusion matrix of \(m\) parallel queries factors as \(Q_1 \otimes \cdots \otimes Q_m\). The serial product needs only chain-stage independence; the Kronecker product needs Axiom 1.

3.5 Miller-Rabin

Miller-Rabin primality testing is a \(k\)-fold one-sided \(\Bernoulli {\BoolType }\) conjunction over a single latent bit, the primality of \(n\). Number theory contributes one fact, the per-round bound \(\fprate \le 1/4\); everything else is the composition machinery of Section 3.1 through Section 3.4.

The witness test as a noisy bit

Fix an odd \(n > 3\) and write \(n - 1 = 2^r d\) with \(d\) odd. A witness \(a \in \{2, \ldots , n-2\}\) certifies compositeness if the sequence \(a^d, a^{2d}, \ldots , a^{2^r d} \pmod {n}\) reaches \(1\) without passing through \(-1\) and without starting at \(\pm 1\); otherwise the test reports “probably prime.” Identify “probably prime” with \(\top \) and “composite” with \(\bot \), and let the true value be whether \(n\) is prime. Then a prime always passes (by Fermat’s little theorem and the fact that the only square roots of \(1\) modulo a prime are \(\pm 1\)), so the test has no false negatives, \(\fnrate = 0\). A composite passes only when \(a\) lies in its strong-liar set, the complement of its witnesses, and Rabin’s theorem bounds that set at \((n-3)/4\), so a uniform witness gives \begin {equation} \label {eq:mr-fpr} \fprate \;\le \; \frac {1}{4}. \end {equation} A single round is therefore a \(\Bernoulli {\BoolType }\) instance with \(\fnrate = 0\) and \(\fprate \le 1/4\).

To bring the \(k\)-round test under Axiom 1 ( Axiom  1), recast it as a predicate on the witness pool: \(\tilde {P}(a)\) is the verdict for witness \(a\), the pool \(\{2, \ldots , n-2\}\) is the universe, and distinct witnesses are distinct input elements. The \(k\)-round test is the conjunction \(\tilde {P}(a_1) \wedge \cdots \wedge \tilde {P}(a_k)\) over \(k\) independently drawn witnesses, exactly the operation of Section 3.1. Because every round reads the same latent value, this is the shared-truth conjunction discussed after Proposition 3.2.1, which is a two-rate instance; the latent-assignment dependence of Section 3.3 does not arise.

The k-round bound

Each round is one-sided with \(\fnrate _i = 0\), so the conjunction has \(\fnrate _k = 0\): the test never calls a prime composite. All \(k\) rounds test the same \(n\), so the only true input is \(\bot \) (for a composite), and the false-positive rate is the all-negative dual of the composition theorem, (3.7): the conjunction false-positives only if every witness is a strong liar, so \begin {equation} \label {eq:mr-k-round} \fprate _k \;=\; \prod _{i=1}^{k} \fprate _i \;\le \; \left (\frac {1}{4}\right )^k. \end {equation} The equality is the product over the independently drawn witnesses (membership of a uniform draw in the strong-liar set is independent across draws); the inequality bounds each round by the Monier-Rabin value \(\fprate _i \le 1/4\). Both ingredients are needed: the size bound \(|L_n| \le (n-3)/4\) from number theory and the independent sampling that makes Axiom 1 apply. At \(k = 5\) the bound is below \(10^{-3}\), and at \(k = 20\) below \(10^{-12}\), effectively definitive.

A worked composite: \(n = 341\)

The number \(341 = 11 \times 31\) is a Fermat pseudoprime to base \(2\) (\(2^{340} \equiv 1 \pmod {341}\)), so the Fermat test passes it, but Miller-Rabin does not. With \(n - 1 = 2^2 \cdot 85\), take \(a = 2\): \(2^{85} \equiv 32 \pmod {341}\) with \(32 \neq \pm 1\), and \(32^2 \equiv 1 \pmod {341}\). A non-trivial square root of \(1\) has appeared, which a prime modulus forbids, so the witness \(a = 2\) certifies \(341\) composite. Miller-Rabin is stronger than Fermat precisely because it inspects the square roots of \(1\) along the way to \(a^{n-1}\), not just the endpoint.

The unification

Miller-Rabin is not a separate technique. The single round is a \(\Bernoulli {\BoolType }\) instance with \(\fnrate = 0\) and \(\fprate \le 1/4\); the \(k\)-round test is the conjunction of Section 3.1; the bound \(\fprate _k \le (1/4)^k\) is (3.9); and the design rule, choose \(k \ge \tfrac {1}{2}\log _2(1/\epsilon ) = \log _4(1/\epsilon )\) to reach false-positive target \(\epsilon \), is that bound solved for \(k\) (\(k = 5\) reaches \(10^{-3}\) and \(k = 20\) reaches \(10^{-12}\), the rounds quoted above). The number theory supplies the per-round \(1/4\); the rest is Boolean algebra over noisy bits. The broader theory of randomized algorithms, including the Las Vegas versus Monte Carlo distinction, is Chapter 11; the companion one-sided example, the Bloom filter, is Section 3.6.

3.6 Composition is the Bridge to Sets

The Boolean algebra of \(\Bernoulli {\BoolType }\) is the foundation for the richer types of Parts II and III, and three concrete bridges carry it there.

BF Bloom filters are \(m\)-fold conjunctions. A Bloom filter reports membership by checking that all \(m\) hash positions are set, an \(m\)-fold AND of one-sided lookups. Each lookup has \(\fnrate = 0\) (a member sets every position) and per-hash false-positive rate \(\fprate _i\). In the all-negative regime of a non-member, the conjunction false-positives only if every probe lands on a set bit, so by the dual (3.9) of the composition theorem the filter’s false-positive rate is \(\prod _{i=1}^{m} \fprate _i\); conditionally on a realized array with a fraction \(p\) of bits set and independent uniform probes it is exactly \(p^m\). Averaged over the random array this conditional rate gives the classical Bloom formula only approximately, since the fill fraction is itself random [Bos+08; CRJ10]; Chapter 5 treats the distinction. The structure is identical to Miller-Rabin; only the source of the per-stage rate differs, a design parameter here, a number-theoretic theorem there. Chapter 5 develops the filter and its parameter selection in full.

BF Set operations are Boolean operations on membership. For approximate sets \(\tilde {A}, \tilde {B}\), membership \(x \in \tilde {A}\) is a \(\Bernoulli {\BoolType }\) instance, and the set operations are pointwise Boolean operations on these: intersection is AND, union is OR, complement is NOT. The rate formulas of Section 3.1 and Section 3.2 therefore give, per element and per latent pair of memberships, \[ \fnrate _\cap = 1 - (1 - \fnrate _A)(1 - \fnrate _B), \quad \fprate _\cup = \fprate _A + \fprate _B - \fprate _A \fprate _B, \quad (\fprate _{A^c}, \fnrate _{A^c}) = (\fnrate _A, \fprate _A). \] The intersection’s false-positive rate is the product \(\fprate _A \fprate _B\) for elements outside both sets (the bulk regime), and is boundary-dependent for elements in exactly one set, exactly the latent-pair dependence of the AND in (3.2): an approximate intersection has more than one false-positive rate, while an approximate union of one-sided sets keeps a single one (Proposition 3.2.1). Chapter 6 carries the general case, including the rate-uniformity conditions an approximate set must satisfy for these per-element rates to define a single \(\Bernoulli {\Set {S}}\) instance, conditions that chapter 16 (Chapter 16) shows can fail.

BF The Boolean case is a fiber of the map algebra. Replacing the Boolean codomain with an arbitrary type \(Y\) lifts the same composition law: an approximate map \(\tilde {f} : X \to Y\) has a channel matrix that grows from \(2 \times 2\) to \(|Y| \times |Y|\), and for a single input the end-to-end channel of a composition is still the matrix product. The joint behavior across inputs is another matter: when the exact inner map sends two inputs to one intermediate value, a frozen error of the outer map at that value is shared by both composed outputs, so their errors are dependent even though each marginal is right. The Boolean algebra of this chapter is the \(Y = \BoolType \) fiber of the map algebra that Chapter 9 develops, and the latent-assignment caveat of Section 3.3 carries over to it.

The Notes and Lineage follow. Chapter 4 formalizes the independence axioms these derivations assumed, and Part II builds the full algebra of approximate sets and maps on both chapters.

3.7 Notes and Lineage

BF Lineage. Forward propagation of error probabilities through AND, OR, and NOT is old. The read-once rules of Proposition 3.3.1 are the signal-probability calculus of Parker and McCluskey [PM75], which already handled asymmetric inputs; probabilistic transfer matrices [Kri+08] give the matrix form for whole circuits, including reconvergent fan-out. Reliable computation from noisy gates goes back to Neumann [Neu56], with sharp formula-size results by Pippenger [Pip88] and Evans and Schulman [ES99]. The parity identity of Remark 3.3.1 is Lemma 1 of Gallager [Gal62] and Matsui’s piling-up lemma [Mat94]. The composition theorem is the product rule for a series system. Miller-Rabin has two parents: Miller [Mil76] gave the witness structure under the Extended Riemann Hypothesis, and Rabin [Rab80] randomized the witness and proved the unconditional \(1/4\) bound; Cormen et al. [Cor+09, Ch. 31] and Mitzenmacher and Upfal [MU17] give textbook treatments.

BF What this chapter adds. A uniform reading of these gates as operations on two-rate atoms, and a correction. The quantum-readout manuscript of this program [Tow26a] asserted that the pair \((\fprate , \fnrate )\) is carried exactly through any Boolean tree. It is not: a gate over distinct latent bits has one rate on one side and up to three on the other (Proposition 3.2.1), and Example 3.1 shows the summary halving the true error. What survives is exact per-latent-assignment propagation, prior-averaged rates that compose only for read-once formulas, and the recursive worst-case bounds of Proposition 3.3.2. The composition paper of the program [Tow26f] handles the set-level version correctly, by letting composition raise the model order.

BF Left out. Fault-tolerant constructions, threshold results for noisy formulas, and the complexity of exact evaluation for formulas with many repeated variables (where the Shannon expansion is exponential) are not treated; the references above cover them. Correlated gate errors are deferred to chapter 16.

Chapter 4
Two Axioms and Kronecker Factorization

4.1 Why Axioms?

Chapters 1 through 3 derived the gate rates, the per-latent-assignment propagation of Proposition 3.3.1, the composition theorem (Section 3.4), and the Miller-Rabin bound, and at two points they used independence without naming it. This chapter names the two assumptions, states them precisely, and proves what their conjunction buys: the Kronecker factorization that collapses the parameter count of an \(m\)-query analysis from exponential to linear, and the formal definition of the type the first three chapters circled.

The first assumption is element-wise independence: error events at distinct inputs are independent. Every product of probabilities in chapters 1 through 3, the composition theorem’s chain, Miller-Rabin’s \(k\) independent witnesses (Section 3.5), the AND and OR truth tables of Section 3.1, rested on it. When it fails, as for two classifiers trained on shared data whose errors correlate, the product formulas become bounds at best. We call it Axiom 1 (Section 4.2).

The second is rate stability: each block’s error rate is a fixed property, the same on the hundredth query as the first. The AND derivation treated \(\fprate \) and \(\fnrate \) as constants, assuming the channel does not drift, warm up, or adapt. We call it Axiom 2 (Section 4.3). It rules out fatigue, history-dependence, and adversarial adaptation, and it is a real restriction, not a triviality. Together the two axioms force the joint confusion matrix of \(m\) queries to factor as a Kronecker product of \(m\) small per-query matrices (Section 4.4), after which the formal definition of Section 4.5 writes itself: a triple \((X, Y, Q)\) satisfying the two axioms, of which every example in chapters 1 through 3 is an instance.

4.2 Axiom 1: Element-Wise Independence

The first axiom governs the spatial structure of error: how the outcomes at distinct inputs relate. For an objective set \(A \subseteq U\) and the approximate predicate \(\tilde {A}\), write the error indicator \(E_x = \mathbf {1}[ \tilde {A}(x) \neq \mathbf {1}_A(x)]\) at each \(x \in U\).

This is a property of the process generating the answers, not of any particular implementation, and its immediate consequence is that joint query distributions factor.

This proposition is the engine behind every multi-query formula in the book: a joint probability over distinct inputs written as a product of marginals is an application of it. It is also what the Bloom filter’s analysis assumes (distinct non-members are independent false positives) and what Miller-Rabin’s \(k\)-round bound needs (distinct witnesses are independent error events, Section 3.5). For the Bloom filter the axiom holds conditionally on the realized bit array, under the random-oracle model (Chapter 5); unconditionally, distinct non-members are positively correlated through the shared fill of the array, the Jensen gap of Corollary 4.4.1.1. The conditioning is part of the axiom: it is stated given the error rates.

Axiom 1 is a genuine modeling assumption, not an automatic consequence of each marginal being well-formed. Two classifiers trained on shared data are each a \(\Bernoulli {\BoolType }\), but their errors on a single negative input correlate. If their error indicators have correlation \(\rho \), the joint false-positive probability is \[ \Prob {E^{(1)} = 1, E^{(2)} = 1} = \fprate _1 \fprate _2 + \rho \sqrt {\fprate _1(1 - \fprate _1)\,\fprate _2(1 - \fprate _2)}, \] which for \(\fprate _1 = \fprate _2 = 0.10\) and \(\rho = 0.60\) is \(0.064\), a factor of \(6.4\) above the product \(0.01\) that independence would predict. The marginals can be perfectly calibrated while the joint fails to factor; correlated errors are worse than independent ones in this precise, quantifiable sense. (The two-classifiers-on-one-input case is one step removed from Axiom 1, which governs distinct inputs to one predicate, but the arithmetic of the violation is identical.) The Kronecker theorem of Section 4.4 makes this exact at the level of the joint confusion matrix: with Axiom 1 the matrix has a structured product form, and without it the parameter count explodes.

4.3 Axiom 2: Conditional Independence of Block Error Rates

Axiom 1 governs error across distinct elements; Axiom 2 governs the random per-block rates. The model partitions the input domain into blocks \(B_1, \ldots , B_n\) sharing a rate each (Section 2.4); for the Boolean domain the blocks are \(B_1 = \{\top \}\), carrying \(\fnrate \), and \(B_2 = \{\bot \}\), carrying \(\fprate \).

Knowing the error rate on one block tells you nothing about another’s. In the order-2 Boolean case this is the single statement that \(\fprate \) and \(\fnrate \) are conditionally independent given \(A\), the instance that arises throughout the book. The rates are random in the sense that they may vary across realizations of the data structure (a Bloom filter’s false-positive rate depends on which elements were inserted); Axiom 2 constrains how those rates relate across blocks, not across elements within a block, which is Axiom 1’s identically- distributed clause.

This is the precise sense of “one channel matrix \(Q\) per block”: the entry \(Q_{ij}\) does not drift or warm up, and a single \(Q\) describes an entire query sequence. A channel that violates this is a Markov-correlated one, where the rate after an error rises from a base \(\fprate _0\) to an elevated \(\fprate _1\). With \(\fprate _0 = 0.05\) and \(\fprate _1 = 0.30\), two consecutive negatives are both reported \(\top \) with probability \(0.05 \times 0.30 = 0.015\), six times the \(\fprate _0^2 = 0.0025\) that a stable rate predicts. Error clustering of this kind is invisible to the factored model and is exactly what Axiom 2 forbids; a Markov channel must be handled by augmenting the state, after which the standard machinery applies to the augmented model.

Stability is what makes the Kronecker factorization possible (Section 4.4): the \(m\)-query joint matrix is a product of copies of one stable \(Q\). The payoff is parametric. A general order-\(n\) channel on \(m\) joint queries needs \(n(2^m - 1)\) parameters in the per-source-block convention; under the two axioms the Kronecker structure forces \(nm\), with no cross-query terms. For the order-2 Boolean channel at \(m = 50\) that is \(2m = 100\) parameters rather than \(2(2^{50} - 1) \approx 2.25 \times 10^{15}\). (One subtlety, taken up in Section 4.4: this count is exact for the rate-conditional matrix always, and for the unconditional matrix when rates are deterministic or no block is queried twice, Corollary 4.4.1.1; same-block queries on genuinely random rates incur a Jensen gap.)

4.4 The Kronecker Factorization Theorem

The two axioms together force a precise structure on the joint behavior of many queries. Index the \(m\) queries by position \(\ell \), with query \(\ell \) on an element of block \(B_{j_\ell }\) producing output \(k_\ell \). The Kronecker product \(P \otimes Q\) replaces each entry \(P_{ij}\) by the scaled block \(P_{ij} Q\); it is associative. The theorem below is the familiar fact that independent components form a product channel, a memoryless channel in information theory and a tensor-product noise model in quantum readout; it is stated here with the conditioning on random rates made explicit, because the structures of later chapters have random rates.

The conditional factorization holds always; the unconditional one needs the distinct-block hypothesis because taking the expectation through the product requires independent factors. When two queries hit the same block \(B_j\), both involve the same random \(\alpha _j\), and \(\Expect {\alpha _j^2} > (\Expect \alpha _j)^2\) opens a Jensen gap, so the unconditional joint exceeds the Kronecker prediction: a mixture of product laws is not a product law. Two regimes avoid this: deterministic rates (where \(Q^*_\ell = Q_\ell \) and the conditional and unconditional forms coincide, the case of every worked example in this book) and pairwise-distinct queried blocks.

The bound is tight: \(m\) queries with independently chosen rates \((\fprate _\ell , \fnrate _\ell )\) satisfy both axioms, and distinct parameter tuples give distinct joint matrices, so no reparameterization beats \(2m\). None of this is deep: the conditional form is the definition of conditional independence, the unconditional corollary is the expectation of a product of independent factors, and the Jensen gap is the positive correlation of any mixture of independent trials. The source manuscript [Tow26b] records the factorization as a remark for deterministic or conditioned rates; the book separates the conditional and unconditional statements because the distinction decides whether a Bloom filter’s false positives on distinct non-members can be multiplied (Section 4.2).

The BSC, queried m times

The binary symmetric channel of Section 2.1 is the cleanest instance. It has one crossover parameter \(\fprate = \fnrate \), so its two inputs share a rate and merge into a single block: the BSC is model order 1 (Section 2.4), studied as the symmetric constrained member of the order-2 family. In the deterministic-rates regime, \(m\) independent queries through it have joint matrix \(Q^{\otimes m}\). For \(m = 2\), \[ Q^{\otimes 2} = \begin {pmatrix} (1-\fprate )^2 & (1-\fprate )\fprate & \fprate (1-\fprate ) & \fprate ^2 \\ (1-\fprate )\fprate & (1-\fprate )^2 & \fprate ^2 & \fprate (1-\fprate ) \\ \fprate (1-\fprate ) & \fprate ^2 & (1-\fprate )^2 & (1-\fprate )\fprate \\ \fprate ^2 & \fprate (1-\fprate ) & (1-\fprate )\fprate & (1-\fprate )^2 \end {pmatrix}, \] each entry a product of two transition probabilities. The general order-2 channel \(Q = \left (\begin {smallmatrix} 1-\fnrate & \fnrate \\ \fprate & 1-\fprate \end {smallmatrix}\right )\) verifies the same way: the \((B_1 B_1, {++})\) entry of \(Q \otimes Q\) is \(\tprate ^2\), which is \(\Prob {\text {both } \top \mid \text {both positive}} = \tprate \cdot \tprate \) by Axiom 1, with each \(\tprate \) the same stable value by Axiom 2. The two-query joint thus carries \(4\) parameters, not the \(2(2^2 - 1) = 6\) a general channel would.

This is the theorem the AND derivation of Section 3.1 tacitly used: the two operands’ joint channel is \(Q_1 \otimes Q_2\), and the AND output is a deterministic function of that joint, with Axiom 1 supplying the factorization and Axiom 2 the stable rates. It also explains the caveat of Section 3.3: the joint channel has four rows, one per latent pair, and a deterministic function of it inherits four rows, not two. Serial composition (matrix product) and this Kronecker factorization (tensor product) answer different questions in different geometries: serial composition threads one element through a chain and needs only the law of total probability, while the Kronecker product queries \(m\) elements in parallel and needs the axioms. With the theorem in hand, the formal definition of Section 4.5 is parametrized by finitely many numbers, two in the Boolean case, rather than an exponential family of joint distributions.

4.5 The \(\Bernoulli {\BoolType }\) Type, Formally

The two axioms and the Kronecker theorem earn the formal definition the first three chapters circled.

We write \(\Bernoulli {\BoolType }\) for the type, the collection of all such triples; a predicate or data structure is an instance when its behavior matches the definition. The pair \((\fprate , \fnrate )\) identifies \(Q\) within the type. Axiom 1 is what lets \(Q\) be read as a per-element description (without it, cross-input correlation needs a richer object), and Axiom 2 is what makes \(Q\) time-invariant; together they are the content of Corollary 4.4.1.2. The definition is for the Boolean case; the extension to \(\Bernoulli {T}\) for general \(T\) is Part III.

Example 4.1 (Biased coin) A coin lands heads (\(\top \)) with prior \(p\), observed through a channel with rates \((\fprate , \fnrate )\). The joint distribution over (truth, report) pairs is \(p\,\tprate \), \(p\,\fnrate \), \((1-p)\,\fprate \), \((1-p)\,\tnrate \) for TP, FN, FP, TN, summing to one. The type characterizes the channel, not the source: the prior \(p\) is not part of \((X, Y, Q)\), though it enters when computing outcome probabilities. At \(p = 0.6\), \(\fprate = 0.05\), \(\fnrate = 0.10\) the four outcomes are \(0.540, 0.060, 0.020, 0.380\), and \(Q = \left (\begin {smallmatrix} 0.90 & 0.10 \\ 0.05 & 0.95 \end {smallmatrix}\right )\) is a well-formed \(\Bernoulli {\BoolType }\) instance. Axiom 1 is vacuous for a single query; Axiom 2 is consistent with choosing \(\fprate \) and \(\fnrate \) without coupling.

Chapters 1 through 3 as instances

The definition names what the first three chapters were doing. The three faces of Section 1.1 are all \(\Bernoulli {\BoolType }\): the biased coin above, the binary symmetric channel of Section 2.1 with \(\fprate = \fnrate \), and the classifier verdict with rates set by its decision boundary. The asymmetric channel of Section 2.2 is the general \(Q\); the Z-channel and the Bloom filter are the one-sided instance \(\fnrate = 0\).

Only NOT among the gates of chapter 3 is an operation on the type: it is the unary \(\Bernoulli {\BoolType } \to \Bernoulli {\BoolType }\) that swaps \((\fprate , \fnrate )\). AND and OR over distinct latent bits do not return an instance. The AND output has the single false-negative rate \(1 - (1 - \fnrate _1)(1 - \fnrate _2)\) but three negative rows, \(\fprate _1\fprate _2\), \((1 - \fnrate _1)\fprate _2\), and \(\fprate _1(1 - \fnrate _2)\), so as a channel from the latent pair it has more than two blocks and is not a triple \((X, Y, Q)\) of the definition above (Proposition 3.2.1); OR is dual. A two-rate description of a gate or formula output exists in three restricted senses only: when all operands read one latent bit (the Miller-Rabin regime), when the operands are one-sided in the direction the gate preserves (OR of \(\fnrate = 0\) instances, AND of \(\fprate = 0\) instances), or as a summary that is either a prior average or a worst-case bound (Remark 3.3.2, Proposition 3.3.2). The exact object is the error at each latent assignment (Proposition 3.3.1). Serial composition of channels (matrix product) and the Kronecker factorization (parallel, tensor product) are different constructions: the first needs only the law of total probability, the second needs the axioms.

The atom-outward order was deliberate: examples first, axioms next, the formal definition last, so that it arrives already supported by intuition. The reader who followed chapters 1 through 3 has watched it emerge. Section 4.6 looks ahead to Part II.

4.6 What Comes Next

Part I is complete. The two axioms are stated (Section 4.2, Section 4.3), the Kronecker theorem shows what they license (Theorem 4.4.1), and Definition 4.5.1 gives \(\Bernoulli {\BoolType }\) a rigorous identity, of which every example in chapters 1 through 3 is an instance. Three of the framework’s organizing principles are now in view, each with its scope: the two axioms; closure of binary channels under serial composition and Kronecker factorization of parallel queries, but not closure of the two-rate type under Boolean formulas, where the exact description is per latent assignment (Section 3.3); and the Boolean thread that ran from the noisy bit to the formal definition without leaving \(\{\top , \bot \}\). Two more arrive in Part II: parametric parsimony, already counted in Corollary 4.4.1.2 and priced in bits by the entropy bounds of chapter 8, and space-accuracy duality, the trade-off between a structure’s size and its error rates, which is Part II’s central concern.

Part II asks what a set-membership predicate built from atoms looks like. A set \(S\) over a universe induces a Boolean query per element; an approximate set makes each query a \(\Bernoulli {\BoolType }\) instance, mutually independent by Axiom 1 given the realized rates. The canonical construction is the Bloom filter, a mechanism for manufacturing one-sided instances (\(\fnrate = 0\)) with a tunable false-positive rate, developed in Chapter 5; chapter 6 then propagates error through set operations, chapter 7 develops the classification measures, and chapter 8 derives the lower bounds that make space-accuracy duality precise. The Boolean thread becomes the foundation, one \(\Bernoulli {\BoolType }\) instance at a time.

4.7 Notes and Lineage

BF Lineage. The Kronecker product and its algebra (mixed products, vectorization) are standard [HJ91, Ch. 4]. That independent components form a product channel, whose transition matrix is the tensor product of the per-component matrices, is the memoryless (extension) channel of information theory [CT06, Ch. 7], and it is the tensor-product noise model of quantum readout-error mitigation, where per-qubit \(2 \times 2\) response matrices are combined by Kronecker products [Bra+21; Smi+21]. The Jensen gap for random rates is the positive correlation of a mixture of independent trials. The two-axiom formulation, and the counting that calls the collapse from an exponential to a linear number of parameters “parametric parsimony,” come from the source manuscripts [Tow26b; Tow26k].

BF What this chapter adds. Presentation and scope. The axioms are separated and stated conditionally on the realized rates, the conditional and unconditional forms of the factorization are distinguished (Theorem 4.4.1, Corollary 4.4.1.1), and the distinction is tied to the structures that need it: a Bloom filter satisfies Axiom 1 only given its array. The formal type is defined so that its scope is visible: NOT is an operation on it, while AND and OR over distinct latent bits leave it (Proposition 3.2.1, Section 3.3).

BF Left out. Dependent errors are not modeled here; Markov-correlated channels are mentioned only to show what Axiom 2 excludes, and chapter 16 treats correlated and adaptive errors. Identifiability and estimation of the \(2m\) parameters from data are not developed beyond the single-query estimators of chapter 1.

Part II
From Bits to Sets

Chapter 5
Approximate Sets and Predicates

5.1 From Atom to Set

Part I studied a single noisy bit. Part II takes many of them, arranges them into a structure, and studies the algebra of that structure. The first such structure is the approximate set, and it is forced rather than invented.

A classical set \(A \subseteq \Universe \) is characterized entirely by its membership predicate \(\SetIndicator {A} : \Universe \to \{\top , \bot \}\), which returns \(\top \) on members and \(\bot \) elsewhere; two sets are equal exactly when their predicates agree everywhere. The predicate is the set, and set-builder notation \(\{x : \SetIndicator {A}(x) = \top \}\) is the predicate read backwards. Replace the exact predicate with a noisy one and each query \(\ASet {A}(x)\) becomes an instance of the formal type \(\Bernoulli {\BoolType }\) of Definition 4.5.1: a Boolean that is usually, but not always, the correct verdict. There is nowhere else for the noise to live, because the membership test is the only behavioral handle on a set.

The objective set \(A\) is the truth; \(\ASet {A}\) is its noisy report. The model never loses \(A\): the rates of \(\ASet {A}\) are defined against it. What a query loses is the ability to read \(A\) off directly. The structure may be a Bloom filter, a perfect hash filter, or a thresholded classifier; what makes it an approximate set in the sense of this book (a Bernoulli set) is that, under the axioms of chapter 4, each query is governed by one \(2 \times 2\) channel matrix, two rates of error and nothing else. Read element by element, this is the misclassification model of a binary measurement and, for an indicator, the randomized-response channel; Section 5.6 traces that lineage.

BF The two rates. Every query is one of two kinds. For \(x \notin A\) the correct verdict is \(\bot \), and the only error is reporting \(\top \); for \(x \in A\) the correct verdict is \(\top \), and the only error is reporting \(\bot \). These define the false-positive and false-negative rates, \[ \fprate = \Prob {\ASet {A}(x) = \top \Given x \notin A}, \qquad \fnrate = \Prob {\ASet {A}(x) = \bot \Given x \in A}, \] the same two rates a single noisy bit carried in Section 1.1, now read at the level of a set. The description does not grow with the set: one \(\fprate \) governs every non-member query and one \(\fnrate \) every member query, no matter how large \(|A|\) or \(|\Universe |\).

BF The block structure. That collapse from \(2|\Universe |\) potential per-element rates to two is not automatic; it is Axiom  2 (Axiom 2) at work. Membership partitions the universe into the in-set block \(B_1 = A\) and the out-of-set block \(B_2 = \Universe \setminus A\), and the axiom forces one rate per block. An approximate set is thus a \(\Bernoulli {\BoolType }\) instance whose input domain is \(\Universe \) rather than \(\{\top , \bot \}\), with the membership partition standing in for the Boolean one. The governing matrix is the chapter-4 channel, \[ Q \;=\; \begin {pmatrix} 1 - \fnrate & \fnrate \\ \fprate & 1 - \fprate \end {pmatrix}, \] row \(1\) for the in-set block, row \(2\) for the out-of-set block. Many queries inherit the chapter-4 machinery wholesale: Theorem 4.4.1 factors the joint behavior of \(n\) distinct queries as the Kronecker product of their per-query channels, using Axiom 1 ( Axiom  1) for cross-element independence and Axiom 2 for the shared per-block rate. The full joint description of an \(n\)-query workload costs two numbers, not \(2^n\). Section 5.3 makes the transfer precise; Section 5.2 first builds the canonical instance, the Bloom filter, in closed form, and Section 5.4 draws the line between approximate sets with one structural zero rate and those with two positive rates.

5.2 The Bloom Filter From Scratch

Section 5.1 defined the approximate set abstractly and said nothing about building one in finite space. The Bloom filter is the canonical construction: a bit array plus \(k\) hash functions, with a false-negative rate that is structurally zero, a false-positive rate in closed form, and a design knob whose optimal setting we derive. Every later chapter of Part II returns to it.

The construction

Fix a universe \(\Universe \), a bit count \(m\), and a hash count \(k\). The structure is a bit array \(b[1\,..\,m]\), initialized to zero, with \(k\) hash functions \(h_1, \ldots , h_k : \Universe \to \{1, \ldots , m\}\). The analysis treats each \(h_i\) as a uniform random function and the \(k\) of them as mutually independent; real implementations approximate this with families such as double hashing over a single digest, which Kirsch and Mitzenmacher [KM08] show leaves the asymptotic false-positive rate unchanged, so the divergence is ignored here.

Both operations run in \(O(k)\) time, independent of \(|A|\) and \(|\Universe |\), and the array is the only state. We follow the literature’s \((m, k, n)\) convention, with \(n = |A|\) the item count; where a later chapter risks confusing this with a workload size we write \(n_{\text {items}}\).

The false-negative rate is structurally zero

If \(x \in A\) was inserted, the bits \(b[h_1(x)], \ldots , b[h_k(x)]\) were set to \(1\) at insertion, and no operation ever resets a bit. Every later Query\((x)\) therefore reads \(k\) ones and returns \(\top \). The argument is structural: it uses only the monotonicity of the array, not the hash family, the load, or the workload. So \[ \fnrate \;=\; \Prob {\textsc {Query}(x) = \bot \Given x \in A} \;=\; 0 \] for every parameter setting. A Bloom filter is one-sided (Section 5.4): one of its two rates is driven to zero by construction rather than by tuning. The remaining question is the size of the other.

The false-positive rate

A query for \(x \notin A\) returns \(\top \) iff all \(k\) bits \(b[h_i(x)]\) are \(1\). Fix a position \(j\). Each of the \(kn\) hash values produced during insertion misses \(j\) with probability \(1 - 1/m\), so \[ \Prob {b[j] = 0} \;=\; \left (1 - \tfrac {1}{m}\right )^{kn} \;\approx \; e^{-kn/m}, \] using \(\ln (1 - 1/m) \approx -1/m\), with error of order \(1/m^2\) per insertion. Write \(p = 1 - e^{-kn/m}\) for the resulting bit-occupancy probability; the load \(kn/m\) is the average number of hash hits per bit. A query reads \(k\) bits, each \(1\) with marginal probability \(p\). Treating the \(k\) reads as independent gives the standard approximation, the working formula of the literature, \begin {equation} \fprate \;\approx \; \left (1 - e^{-kn/m}\right )^k. \label {eq:bloom-fpr} \end {equation}

Optimal \(k\) for fixed \(m\) and \(n\)

The bit count \(m\) is the space budget and \(n\) is fixed by the application; the hash count \(k\) is the design knob. Let \(f(k) = (1 - e^{-kn/m})^k\), minimized over \(k > 0\) as a continuous variable. Taking logarithms and differentiating, \[ \frac {d}{dk}\ln f(k) \;=\; \ln (1 - e^{-kn/m}) \;+\; \frac {kn}{m}\cdot \frac {e^{-kn/m}}{1 - e^{-kn/m}}. \] Writing \(u = e^{-kn/m}\), so that \(kn/m = -\ln u\), the first-order condition \(\ln (1-u) - (\ln u)\,\tfrac {u}{1-u} = 0\) becomes, after multiplying by \((1-u)\), \begin {equation} (1 - u)\ln (1 - u) \;=\; u \ln u. \label {eq:bloom-foc} \end {equation} By inspection \(u = \tfrac 12\) solves (5.2), both sides equal \(-\tfrac 12\ln 2\). It is the only interior solution. Set \(\phi (u) = (1-u)\ln (1-u) - u\ln u\) on \((0,1)\); it is antisymmetric about \(\tfrac 12\), \(\phi (1-u) = -\phi (u)\), so \(\phi (\tfrac 12) = 0\) and \(\phi \to 0\) at both endpoints. Its derivative \(\phi '(u) = -\ln \!\big (u(1-u)\big ) - 2\) is symmetric about \(\tfrac 12\), diverges to \(+\infty \) at the endpoints, and dips to \(\phi '(\tfrac 12) = 2\ln 2 - 2 < 0\), so \(\phi '\) changes sign exactly twice. Hence \(\phi > 0\) on \((0, \tfrac 12)\) and \(\phi < 0\) on \((\tfrac 12, 1)\), vanishing only at \(u = \tfrac 12\). The boundary behavior pins this critical point as the minimum: \(f \in (0,1)\) in the interior and \(f \to 1\) at both endpoints (as \(k \to 0^+\), \(\ln f \approx k\ln (kn/m) \to 0\); as \(k \to \infty \), \(\ln f \approx -k\,e^{-kn/m} \to 0\)), and the second-derivative check at \(u = \tfrac 12\) gives \(\tfrac {d^2 \ln f}{dk^2} = 2(n/m)(1 - \ln 2) > 0\). Substituting \(u = \tfrac 12\) into \(kn/m = -\ln u\), \begin {equation} k^* \;=\; \frac {m}{n}\ln 2, \label {eq:bloom-optimal-k} \end {equation} at which each bit is occupied with probability exactly \(\tfrac 12\) and \begin {equation} \fprate ^* \;\approx \; \left (\tfrac 12\right )^{k^*} \;=\; 2^{-(m/n)\ln 2} \;=\; (0.6185\ldots )^{m/n}. \label {eq:bloom-optimal-fpr} \end {equation}

The bit-per-item ratio \(m/n\) is the right measure of a Bloom filter’s space budget: at the optimal \(k\) the rate decays exponentially in \(m/n\) with base \(0.6185\), so each additional bit per item divides the rate by about \(1.62\), and halving the rate costs \(\log _2 e \approx 1.44\) bits per item. Chapter 8 compares this to the information-theoretic floor and finds a fixed gap of \(\log _2 e \approx 1.44\); the half-full array at the optimum is the striking signature of the construction, most accurate not when sparse but when exactly half its bits are set.

A worked example

Take \(m = 10000\), \(n = 1000\), a comfortable \(m/n = 10\) bits per item. Then \(k^* = 10\ln 2 \approx 6.93\), rounded to \(k = 7\), giving \(\fprate ^* \approx 0.6185^{10} \approx 0.0082\), about one false positive per \(120\) negative queries. Evaluating (5.1) around the optimum confirms the rounding and shows how shallow the minimum is:

The rates at \(k = 6, 7, 8\) agree to the third significant figure, and the occupancy at \(k = 7\) is \(0.5034\), half-full as predicted. The minimum sits where two pressures balance: raising \(k\) widens the conjunction the query must satisfy (pushing the rate down) but fills the array faster (pushing it up), and the two cancel at half occupancy. Off the optimum the penalty is real, \(k = 1\) at \(m/n = 10\) gives \(\fprate \approx 0.095\), twelve times the optimum, and the rate depends on \(m\) and \(n\) only through \(m/n\) once \(k\) is optimized.

The Bloom filter is one row of the general predicate algebra: \(\fnrate = 0\) structurally, \(\fprate \) in closed form, the same \(\Bernoulli {\BoolType }\) channel and chapter-4 axioms as any other approximate set. Section 5.3 develops that algebra in generality; Section 5.4 takes up the one-sided property \(\fnrate = 0\) this construction enjoys.

5.3 The Predicate Algebra

Section 5.1 described an approximate set’s rates informally and Section 5.2 built one instance. This section places the abstraction inside the chapter-4 machinery at the matrix level: the per-element channel, the Axiom-2 collapse to two per-block rates, and the Kronecker factorization over many queries. The payoff is the chapter’s first formal statement of parametric parsimony, that an approximate set’s entire joint behavior is fixed by two numbers.

Per-element channels and their collapse

For each \(x \in \Universe \) the query \(\ASet {A}(x)\) is a \(\Bernoulli {\BoolType }\) instance. In the most general setting each \(x\) carries its own rate, \(\fnrate (x)\) if \(x \in A\) and \(\fprate (x)\) if \(x \notin A\), assembling into a per-element channel \[ Q_x \;=\; \begin {pmatrix} 1 - \fnrate (x) & \fnrate (x) \\ \fprate (x) & 1 - \fprate (x) \end {pmatrix}, \] row \(1\) for the in-set case, row \(2\) for the out-of-set case, matching the convention of Definition 4.5.1. Unconstrained, the family \(\{Q_x\}\) carries up to \(2|\Universe |\) parameters. Axiom 2 collapses it.

Throughout Part II we work in the deterministic-rates regime of Chapter 4, where \(\fprate \) and \(\fnrate \) are fixed parameters rather than random variables; the conditional and unconditional Kronecker forms then coincide and we write a single \(Q\) for \(\ASet {A}\). What varies across queries is the row a query reads, not the matrix.

The joint distribution factors via Kronecker

A workload of \(m\) queries on distinct elements \(x_1, \ldots , x_m\), with \(x_\ell \) in block \(j_\ell \) and report \(k_\ell \), has joint behavior governed by Theorem 4.4.1. Its hypotheses are the two axioms: Axiom 1 ( Axiom  1) for cross-element independence and Axiom 2 for the block-stability just established. Since every per-query matrix is the same \(Q\), the joint confusion matrix is the \(m\)-fold Kronecker power.

This earns at the joint level the informal claim of Section 5.1: two scalars fix \(Q\), \(Q\) fixes the Kronecker power, and the power fixes the behavior of arbitrarily many queries. To see it concretely, take \(\Universe = \{1, \ldots , 10\}\), \(A = \{1, 2, 3\}\), \(\fprate = 0.1\), \(\fnrate = 0.05\), so \(Q = \left (\begin {smallmatrix} 0.95 & 0.05 \\ 0.10 & 0.90 \end {smallmatrix}\right )\). The probability that two in-set queries and two out-of-set queries all report \(\top \) (two true positives, two false positives) is the product of the corresponding cells, \[ Q_{1,1}^2\, Q_{2,1}^2 \;=\; (1 - \fnrate )^2 \fprate ^2 \;=\; 0.95^2 \cdot 0.10^2 \;=\; 0.009025, \] while the all-correct outcome has probability \((1-\fnrate )^2(1-\fprate )^2 = 0.731\). Doubling the universe leaves these unchanged; doubling the workload grows the Kronecker power but keeps each entry a product of two-rate factors, one per query. The Bloom filter is the special case \(\fnrate = 0\); any other construction supplies a different \(Q\) but the same algebra. The distinction Section 5.4 draws next is what happens when one of the two parameters is structurally zero.

5.4 One-Sided vs Two-Sided Approximate Sets

The Bloom filter has one error rate exactly zero, by construction rather than tuning. That is one shape an approximate set can take; others force the false-positive rate to zero on a designated set, or allow both rates to be positive. The shapes are not interchangeable, and the distinction organizes the constructions of Chapter 6, Chapter 14, and Chapter 15.

The name tracks the rate that is zero. A one-sided set sits on a coordinate axis of the rate square \([0,1]^2\); a two-sided set in its interior; the origin \(\fprate = \fnrate = 0\) is the exact set, excluded as not approximate. The Z-channel of Definition 2.2.2 is the per-query analog, and the one-sided property is its per-set lift.

Why FNR-zero is the natural default

The Bloom filter’s FNR-zero property came for free from monotonicity: the insert primitive only sets bits, so a query for an inserted item always reads ones. Any structure on a monotone insert-only primitive shares it, which is why summaries over append-only logs and hash-keyed sketches are FNR-zero for the same structural reason. Forcing the opposite asymmetry is harder. Making \(\fprate = 0\) requires the structure to reject every non-member, that is, to certify membership; a stored copy of the set does this, at a cost of about \(\log _2 |\Universe |\) bits per element, which is the space an approximate set exists to save. A perfect hash with one fingerprint per element does not certify: a member always finds its own fingerprint (the structure is FNR-zero, like the Bloom filter), while a non-member may still collide with the fingerprint stored at its slot. Chapter 15 reaches \(\fprate = 0\) only by restricting the query model so that a non-member cannot present the value it would need to collide. The set must be known up front and updates are expensive; those costs buy certainty in the positive direction.

Safety versus space

One-sidedness is valuable because one direction of the answer is exact: a Bloom filter’s \(\bot \) is certain (the element is definitely absent), so a cache can skip a lookup or a spell-checker can pass a known word without follow-up. An FPR-zero filter has the dual guarantee, a certain \(\top \). Two-sided sets give up the certain direction. Sometimes this is a side effect rather than a goal: a counting Bloom filter [Fan+00] replaces bits with small counters to support deletion, at roughly four times the space, and a counter overflow, or the deletion of an item that was never inserted, can later zero a counter a member still needs, introducing a positive \(\fnrate \) [Guo+10]. Other designs trade false positives for false negatives on purpose: the retouched Bloom filter clears selected bits to remove chosen false positives [DBF06], and the generalized Bloom filter sets and resets bits so that both rates are positive by design [LVD11].

The size of the trade is calculable. By Equation (5.3) a Bloom filter at \(\fprate = 0.01\), \(\fnrate = 0\) needs \(m \approx 1.44\,n \log _2(1/\fprate ) \approx 9.6\,n\) bits. Chapter 8 gives the classical floor for any FNR-zero set at the same \(\fprate \), about \(n\log _2(1/\fprate ) \approx 6.64\,n\) bits, so the Bloom filter pays the familiar factor \(\log _2 e \approx 1.44\). Static constructions from the retrieval literature close most of that gap efficiently (Section 8.4); the Bernoulli Hash Function of Chapter 14 meets the floor in space, but only at a construction cost exponential in \(n\). A two-sided set does not escape the floor by much: tolerating false negatives on a fraction \(\fnrate \) of the members lowers the leading-order cost only to about \((1 - \fnrate )\,n\log _2(1/\fprate )\) bits (Proposition 8.3.1, after Pagh and Rodler [PR01]), so giving up the certain \(\bot \) saves at most about \(\fnrate \log _2(1/\fprate )\) bits per element.

The perfect filter line

The rate plane \((\fprate , \fnrate ) \in [0,1]^2\) parameterizes every approximate set. The one-sided sets occupy the two axes, \(\{\fnrate = 0\}\) and \(\{\fprate = 0\}\); we call their union the perfect filter line, each axis a one-dimensional locus with one rate-degree of freedom. Two-sided sets fill the interior; the exact set is the origin.

The line constrains composition. Chapter 6 derives the union, intersection, complement, and difference of Bernoulli sets and shows:

• the union and the intersection of two FNR-zero sets are FNR-zero;
• the complement swaps the axes, sending FNR-zero to FPR-zero;
• the difference of two sets on the same axis is two-sided, so it leaves the line.

By the complement symmetry the FPR-zero axis obeys the same rules, so the algebra of the line is symmetric. The asymmetry between the two axes is constructive, not algebraic: FNR-zero comes for free from monotone insertion, while FPR-zero needs a certificate. Section 3.6 previewed the per-element union and intersection rates; Chapter 6 derives them in full. Chapter 8 and Chapter 14 then take up the FNR-zero axis (its floor and a construction at it), and Chapter 15 the FPR-zero axis and the price of certifying a subset exactly.

5.5 What Comes Next

The chapter lifted the noisy bit to a function \(\ASet {A} : \Universe \to \Bernoulli {\BoolType }\) (Section 5.1), built the Bloom filter as its canonical realization (Section 5.2), placed both inside the chapter-4 axioms so that two rates fix the entire joint behavior (Section 5.3), and separated the rate shapes an approximate set can take (Section 5.4). The rest of Part II develops the algebra and bounds the construction underwrites. Chapter 6 derives the rates of unions, intersections, complements, and differences from the gate rules of Chapter 3 applied to the predicate algebra, region by region, and asks which subfamilies, the perfect filter line among them, are closed under each operation. Chapter 7 turns the two intrinsic rates into the workload-dependent measures practitioners report, positive and negative predictive value, recall, \(F_1\), and supplies the proper interval estimators that Section 1.4 deferred. Chapter 8 presents the classical floor of \(-\log _2 \fprate \) bits per element for an FNR-zero set, its two-sided form, and the standard Bloom filter’s \(1.44\times \) overhead against it. The two constructions that take the perfect filter line as their subject, Chapter 14 on the FNR-zero axis and Chapter 15 on the FPR-zero side, arrive in Part IV once the intervening theory is in place.

5.6 Notes and Lineage

BF Lineage. The Bloom filter is due to Bloom [Blo70]. Its approximate false-positive formula and the optimal \(k = (m/n)\ln 2\) are standard in the survey and textbook literature [BM04; MU17]; Bose et al. [Bos+08] showed that the formula neglects the dependence between probed bits, Christensen, Roginsky, and Jimeno [CRJ10] gave an exact expression, and Kirsch and Mitzenmacher [KM08] justified double hashing. The space question the filter answers was posed and bounded by Carter et al. [Car+78]. The object the chapter calls an approximate set is older than the filter. A per-element noisy indicator with one false-positive and one false-negative rate is the misclassification model of Bross [Bro54] and, applied to a sensitive indicator, the randomized-response channel of Warner [War65]; Hirota [Hir81] studied probabilistic sets as mathematical objects. Filters with false negatives also exist as constructions: the retouched [DBF06] and generalized [LVD11] Bloom filters are two-sided by design, and the counting filter of Fan et al. [Fan+00] acquires false negatives under counter overflow [Guo+10].

BF What this chapter adds. A presentation rather than a new structure. The Bloom filter is read as one realization of a function \(\Universe \to \Bernoulli {\BoolType }\) and placed under the two axioms, so that the Kronecker factorization of chapter 4 fixes its joint behavior; the one-sided and two-sided vocabulary then organizes chapters 6, 14, and 15. The set-as-predicate reading follows Towell [Tow26b], and the \(\Bernoulli {\BoolType }\) type follows Towell [Tow26k]. The chapter is explicit that a perfect hash with fingerprints is FNR-zero, like the Bloom filter, and that the two axes of the perfect filter line obey the same composition rules.

BF Left out. Dynamic variants beyond the counting filter (scalable, stable [DR06], and deletable filters); cuckoo [Fan+14] and quotient [Ben+12] filters; blocked, cache-efficient layouts; finite-\(m\) exact analysis beyond the smallest example; and adversarially chosen queries, which chapter 16 takes up.

Chapter 6
Composition of Sets

6.1 Union of Approximate Sets

Chapter 5 framed an approximate set as a noisy membership predicate with two per-block rates \((\fprate , \fnrate )\). The first compositional step asks what the union of two such sets is and what rates it inherits. The answer is the composition theorem of Theorem 3.4.1 applied to the OR gate of Section 3.2, lifted from the per-bit level to the per-set level.

To evaluate the union at \(x\), query both operands and OR the verdicts; the rates follow from the operand rates and the OR-gate analysis of Chapter 3.

The asymmetry between the two rates is structural and recurs for every operation in this chapter. The union FPR is a closed-form polynomial in the operand FPRs alone, requiring no prior, because every negative of \(A \cup B\) lies in the single region \(\Universe \setminus (A \cup B)\). The union FNR mixes three per-region rates, because the positives of \(A \cup B\) split into three blocks where the operands play different roles; the composed set is higher-order, and (6.2) reports its marginal FNR over a query distribution. When the operands are FNR-zero (\(\fnrate _A = \fnrate _B = 0\)) every term vanishes and the union stays FNR-zero, the first closure fact of Section 6.4.

In the vocabulary of chapter 2, the union is a fourth-order Bernoulli set: its error rate is constant on each of the four blocks \(A \setminus B\), \(B \setminus A\), \(A \cap B\), and \(\Universe \setminus (A \cup B)\) of the partition the operands generate, and those four block rates, \(\fnrate _A(1 - \fprate _B)\), \(\fnrate _B(1 - \fprate _A)\), \(\fnrate _A\fnrate _B\), and \(1 - (1 - \fprate _A)(1 - \fprate _B)\), are exact. The pair \((\fprate _\cup , \fnrate _\cup )\) is a summary of them. Its first coordinate is exact because the union has a single negative block; its second is a workload-weighted average of three block rates, valid for the stated weights and for no particular element.

BF Two realizations on Bloom filters. Definition 6.1.1 is a predicate-level operation, predicate-OR: keep both filters, query each, OR the outputs; its FPR is (6.1). When both operands are Bloom filters with the same parameters and hash family, a tempting shortcut is bit-OR-Bloom: OR the two bit arrays into one bitmap and query it once. This is a different operation. Bit-OR-Bloom is exactly a Bloom filter holding the union of the inserted items, so its rate is the chapter-5 formula at the combined count, \begin {equation} \label {eq:bit-or-bloom-fpr} \fprate _{\text {bit-OR}} \;\approx \; \bigl (1 - e^{-k(n_A + n_B)/m}\bigr )^k, \end {equation} materially larger than (6.1): at \(m = 5000\), \(k = 7\), \(n_A = n_B = 500\) the operand rate is \(\fprate \approx 0.0082\), predicate-OR gives \(\approx 0.0163\), and bit-OR-Bloom gives \(\approx 0.137\). Both preserve FNR-zero, but they ask different questions. Predicate-OR requires one filter to have all \(k\) bits set; bit-OR-Bloom requires only that at each queried position some filter’s bit is set, with the responsible filter free to vary by position, so it is the more permissive operation and carries the larger FPR. Predicate-OR is the union of Definition 6.1.1; bit-OR-Bloom is a representation choice that trades a higher FPR for one bitmap and one lookup, and it further requires a shared hash family.

For two FNR-zero Bloom filters at \(\fprate _A = 0.01\), \(\fprate _B = 0.02\) the predicate-OR union has \(\fprate _\cup = 1 - (1 - 0.01)(1 - 0.02) = 0.0298\) and \(\fnrate _\cup = 0\). The second-order term \(\fprate _A \fprate _B = 0.0002\) is small, so for small rates the union FPR is nearly additive, \(\fprate _\cup \approx \fprate _A + \fprate _B\), bracketed by \(\max (\fprate _A, \fprate _B) \le \fprate _\cup \le \fprate _A + \fprate _B\).

(6.1) is the composition theorem of Theorem 3.4.1 applied to the OR gate with \(\eta _i = \fprate _i\), the lift to sets that Section 3.6 previewed: the same Boolean operation, the same algebra, with \(\fprate _i\) reread from a per-bit crossover to a per-block set rate. The \(n\)-ary union is the same theorem folded, \(\fprate _\cup = 1 - \prod _i (1 - \fprate _i)\): OR is associative and commutative with identity \(\ASet {\EmptySet }\), so at the level of predicates the union is a monoidal fold, and its false-positive rate folds exactly. Its false-negative rate does not fold as a single number; it must be carried block by block.

Section 6.2 takes up intersection, complement, and difference; Section 6.3 the regimes in which a composition leaves the perfect filter line; Section 6.4 the operations under which each axis of the line is closed.

6.2 Intersection, Complement, Difference

The union of Section 6.1 lifts the OR gate to sets; the remaining Boolean operations lift the same way. Each composed predicate is a Boolean combination of the operand predicates, and its rates follow from the gate analysis of Chapter 3 and cross-set independence. The recurring structure of Proposition 6.1.1 persists: one rate is a clean product over a homogeneous truth-region, the other is a partition-weighted sum over regions where the operands play different roles.

Intersection

The FPR is the dual of the union FNR (6.2), with the same three-region structure: the intersection of two approximate sets is a higher-order Bernoulli set, and (6.5) is its marginal FPR. The bare product \(\fprate _A \fprate _B\) is only the third term, the both-outside region. It is exact when the two operands approximate the same objective set (\(A = B\), so \(A \setminus B = B \setminus A = \varnothing \) and \(w_1 = w_2 = 0\)), and it is the limit when the universe dwarfs both sets (\(w_3 \to 1\)). Outside those regimes the first two terms dominate, and they are linear in a single operand rate, not quadratic: a query landing in \(A \setminus B\) false-positives at rate \(\fprate _B\) alone, because \(\ASet {A}\) accepts the genuine member. The widely quoted “intersection multiplies false-positive rates” holds for filters of one set: intersecting \(k\) independent FNR-zero filters of a common \(A\) drives the FPR to \(\prod _i \fprate _i \to 0\), since on the dominant outside region a negative survives only if every filter false-positives.

Complement

This is the set-level reading of NOT-under-noise from Section 3.2. Complement is an involution exchanging the two coordinate axes of the rate plane; unlike intersection and difference, it needs no partition, because NOT acts pointwise and reclassifies every element by the same rule.

Difference

Both operands’ both rates appear in the difference. The consequence is sharp and is the reason the difference matters for the perfect filter line: even an FNR-zero \(\ASet {A}\) inherits false negatives at rate \(\fprate _B\), because a member of \(A \setminus B\) that \(\ASet {B}\) false-positives is wrongly excluded; and even FNR-zero operands leave a positive difference FPR through the \(w_2 \fprate _A(1 - \fprate _B)\) term, because a point outside both sets that \(\ASet {A}\) false-positives passes the difference (the complemented \(\ASet {B}\) correctly accepts it). The difference of two one-sided sets is therefore two-sided, a fact the worked example makes concrete and Section 6.3 reads as an escape from the line.

Worked example

Take two FNR-zero Bloom filters with \(\fprate _A = 0.01\), \(\fnrate _A = 0\) and \(\fprate _B = 0.02\), \(\fnrate _B = 0\), in a universe large enough that the region outside both sets dominates each partition (\(w_3 \to 1\) for the intersection, \(w_2 \to 1\) for the difference).

Intersection. The FNR is \(1 - (1 - 0)(1 - 0) = 0\): the intersection of two FNR-zero sets stays FNR-zero. The FPR is (6.5), dominated by the outside-both term, \(\fprate _\cap \approx \fprate _A \fprate _B = 0.01 \times 0.02 = 0.0002\), an order of magnitude below either operand. (Were the two sets distinct enough that a query could land in \(A \setminus B\) or \(B \setminus A\), the linear terms \(w_1 \fprate _B + w_2 \fprate _A\) would lift the rate well above the product.) Intersection of same-set filters drives the FPR down; union inflates it; the two are exact duals.

Complement. \(\complement \ASet {A}\) has rates \((\fnrate _A, \fprate _A) = (0, 0.01)\): the FNR-zero false-positive machine becomes an FPR-zero false-negative machine, moved from the horizontal axis to the vertical one.

Difference. Here the products mislead. Naively pairing \(\fprate _A\) with \(\fnrate _B = 0\) would suggest \(\fprate _\setminus = 0\); but that is only the \(w_3\) term of (6.8). The dominant \(w_2\) term survives, \[ \fprate _\setminus \approx \fprate _A (1 - \fprate _B) = 0.01 \times 0.98 = 0.0098, \qquad \fnrate _\setminus = 1 - (1 - 0)(1 - 0.02) = 0.02. \] The difference of two FNR-zero Bloom filters is two-sided, at approximately \((0.0098, 0.02)\), not the FPR-zero point \((0, 0.02)\) the bare product would give. Both failure modes are real: \(\ASet {B}\)’s false positives become wrongful exclusions (the FNR \(= \fprate _B\)), and \(\ASet {A}\)’s false positives on points outside both sets survive the difference (the FPR \(\approx \fprate _A\)). This two-sidedness is the cancellation phenomenon Section 6.3 scrutinizes.

A caveat on bit-AND of Bloom filters

Definition 6.2.1 is a predicate operation: query both filters and AND the verdicts. For two Bloom filters of equal length built with the same hash family, the bitwise AND of the two arrays answers every query exactly as the predicate AND does, since all \(k\) probed bits are set in the AND array iff they are set in both arrays. It is nevertheless not a Bloom filter for \(A \cap B\): a bit set in one array by an element of \(A \setminus B\) and in the other by an element of \(B \setminus A\) survives the AND, so the AND array contains, and is generally fuller than, the array that inserting \(A \cap B\) directly would produce, and its false-positive rate is correspondingly higher. The shared hash family also violates the cross-set independence that Proposition 6.2.1 assumes (Remark 6.1.2), so the proposition describes independently hashed operands, not the bitwise construction; the chapter notebook measures the discrepancy.

6.3 Cancellation and the Loss of One-Sidedness

The operands of Section 6.1 and Section 6.2 were drawn from one-sided sets on the perfect filter line. Some compositions kept the result on the operands’ axis; others did not. This section classifies the three behaviors: a composition can stay on the operands’ axis, walk to the other axis, or leave the line entirely into the two-sided interior. The complement and the mixed-axis intersection drive the last two, and Section 6.4 rereads the classification as a closure question.

Complement swaps which axis

A Bloom filter at \((0.01, 0)\) on the FNR-zero axis has complement \((0, 0.01)\) on the FPR-zero axis: the same information read dually, certain about absence becoming certain about presence. The complement is a discrete reflection across the rate plane, not a continuous deformation; it walks the line from one arm to the other in a single step. Composing union with complement illustrates this without leaving the line. Two FNR-zero filters at \(\fprate _A = 0.01\), \(\fprate _B = 0.02\) have union \((0.0298, 0)\) by Proposition 6.1.1, still FNR-zero, and complementing the union swaps its rates to \((0, 0.0298)\) on the FPR-zero axis. The composed structure stays one-sided; only the certain direction flips, from “is \(x\) in \(A\) or \(B\)?” answered with a sure \(\bot \) to “is \(x\) outside both?” answered with a sure \(\top \).

Mixed-axis intersection leaves the line

The complement enters destructively the moment it sits inside an intersection. Let \(\ASet {A}\) be FNR-zero at \((0.01, 0)\) and \(\ASet {B}\) FNR-zero at \((0.02, 0)\), and form the difference \(\ASet {A} \setminus \ASet {B} = \ASet {A} \cap \complement \ASet {B}\). The complement \(\complement \ASet {B}\) sits at \((0, 0.02)\) on the FPR-zero axis, so the intersection pairs one operand from each axis. By the corrected intersection formula (6.5), the false-positive rate is not the bare product: with \(\ASet {A} = (0.01, 0)\) and \(\complement \ASet {B} = (0, 0.02)\), the outside-both region survives, \[ \fprate _\cap \approx \fprate _A (1 - \fnrate _{\complement B}) = 0.01 \times 0.98 = 0.0098, \qquad \fnrate _\cap = 1 - (1 - 0)(1 - 0.02) = 0.02. \] The composition lands at \(\approx (0.0098, 0.02)\), in the two-sided interior, not on either operand’s axis. This is the same two-sidedness Section 6.2 found for the difference directly, as it must be, since the two are the same set.

The older intuition that this intersection “lands on the FPR-zero axis” is an artifact of approximating \(\fprate _\cap \) by the product \(\fprate _A \fprate _B = 0\); the linear term is what makes the escape immediate, with no need for a leak in either operand’s one-sidedness.

The three regimes

The chapter-6 operations fall into three classes by their action on the perfect filter line.

Axis-preserving. Union and intersection of two FNR-zero sets stay FNR-zero: \(\fnrate = 0\) is preserved (Equation (6.2), Equation (6.4) vanish), even though the surviving FPR is the full three-region rate of §6.2. The FPR-zero side is symmetric.

Axis-swapping. Complement of a one-sided set is one-sided on the other axis (Proposition 6.3.1): the result leaves each axis individually but stays on the line as a whole.

Axis-leaving. Mixed-axis intersection is two-sided (Proposition 6.3.2). Difference \(\ASet {A} \setminus \ASet {B} = \ASet {A} \cap \complement \ASet {B}\) is two-sided exactly when \(\ASet {A}\) and \(\ASet {B}\) lie on the same axis (so the complement sends \(\ASet {B}\) to the opposite axis and the intersection is mixed); it stays one-sided only when the operands lie on opposite axes to begin with. These are the compositions that exit the line into the interior, where both directions of report are uncertain.

Why it matters

Whether a composed set sits on the line governs the three later chapters that take the line as their subject. Chapter 7 simplifies for a one-sided structure (an FNR-zero set has \(\NPV = 1\) and one nonzero rate to estimate) and exercises the full machinery for a two-sided one. Chapter 8 presents the classical floor \(-\log _2 \fprate \) for FNR-zero sets and its two-sided form, and Chapter 14 constructs a structure at the floor; a chain that leaves the axis is priced by the two-sided form instead. Chapter 15 builds on the FPR-zero axis, and a mixed-axis intersection with a Bloom filter destroys its certain \(\top \). Composition discipline, keeping a chain on one axis, is the price of staying near the floor; Section 6.4 states the discipline as a closure result.

6.4 The Perfect Filter Line

Section 5.4 named the perfect filter line as the union of the two coordinate axes of the rate plane. The composition formulas of this chapter, with the cancellation classification of Section 6.3, let us read the line as an algebraic object: a distinguished subset of \([0,1]^2\) that some operations preserve, some map to itself, and some exit.

The axes meet only at the origin \((0,0)\), the exact set, excluded by the strict inequalities: a set on the line achieves certainty in exactly one direction of report, and the origin achieves it in both at the cost of the space approximate sets exist to save. We draw \(\fprate \) horizontal and \(\fnrate \) vertical, so \(\mathcal {L}_{\fnrate }\) is the horizontal arm and \(\mathcal {L}_{\fprate }\) the vertical one; Chapter 14 builds on the horizontal arm and Chapter 15 on the vertical.

Part (4) is the difference’s signature: two FNR-zero operands compose to a two-sided set, because the intermediate complement sends \(\ASet {B}\) to the opposite axis and the intersection then mixes axes. The difference stays one-sided only across axes, when \(\ASet {A}\) and \(\ASet {B}\) start on opposite arms so the complement aligns them; same-axis operands always produce a two-sided difference.

BF The shape of the line. \(\mathcal {L}\) is non-convex: the midpoint of a segment from \((\fprate , 0)\) to \((0, \fnrate )\) is \((\fprate /2, \fnrate /2)\), interior to the square and on neither axis, so even a random choice between one structure from each arm is two-sided. The two arms meet only at the excised origin, so \(\mathcal {L}\) is a wedge of two open segments rather than a connected curve. Algebraically, each axis is closed under union and intersection, complement exchanges the two axes, and mixed-axis intersection and same-axis difference leave the line. The line is therefore not closed under the full set algebra; it is a pair of union-and-intersection-closed families exchanged by complement. Same-axis chains built from union and intersection compose freely and stay one-sided; chains that mix axes leave the line, and the one-sided space accounting of Chapter 8 and Chapter 14 no longer prices the result.

Chapter 14 builds two structures on \(\mathcal {L}_{\fnrate }\): the perfect hash filter, which pays a minimal perfect hash’s overhead above the floor, and the Bernoulli Hash Function, which meets the floor in space at an exponential construction cost; the retrieval-based filters of Section 8.4 remove most of the standard Bloom filter’s \(\log _2 e \approx 1.44\) factor efficiently. By Proposition 6.4.1 unions and intersections of such structures stay on the axis at predictable rates. Chapter 15 builds its rank-table filter on \(\mathcal {L}_{\fprate }\); mixed-axis composition with a Bloom filter leaves the line and forfeits the FPR-zero certainty, so the designer must respect the closure constraints to keep either chapter’s guarantees.

6.5 What Comes Next

The chapter fixed the algebra of set operations: closed-form block rates for union, intersection, complement, and difference (Section 6.1, Section 6.2), with the caveat that a composed set is higher order and its summary pair does not compose further (Remark 6.1.1); the classification of which compositions leave the perfect filter line (Section 6.3); and the operations under which each axis of the line is closed (Section 6.4). Two questions remain for the rest of Part II: how good an approximate set is against a workload, and how small it can be made. Chapter 7 turns the per-block rates into the workload-dependent measures practitioners report, positive and negative predictive value, \(F_1\), recall and precision, feeding the composition rates directly into them (a union of FNR-zero filters is FNR-zero by Proposition 6.4.1, so its PPV follows with no extra work), and it retires the interval-estimator deferral of Section 1.4 with the Wilson, Clopper-Pearson, and Jeffreys constructions. Chapter 8 gives the rates a space-cost analog, the classical floor of \(-\log _2 \fprate \) bits per element for an FNR-zero set and its two-sided form, against which the closure results bound what a composition can cost; Chapter 14 then constructs FNR-zero structures near the floor, the perfect hash filter and the Bernoulli Hash Function, the latter meeting it in space at an exponential construction cost.

6.6 Notes and Lineage

BF Lineage. The rate formulas of Sections 6.1 and 6.2 are the gate rules of chapter 3 applied pointwise to membership, and that computation is old. Forward propagation of signal probabilities through AND, OR, and NOT gates with independent inputs is due to Parker and McCluskey [PM75], and probabilistic transfer matrices [Kri+08] carry the same calculus, asymmetric rates included, through whole circuits. In databases the same rules are the independent-or and independent-and of probabilistic relational algebra [FR97] and of query evaluation on probabilistic databases [DS07]. Bitwise union and intersection of Bloom filters are folklore surveyed by Broder and Mitzenmacher [BM04], and Papapetrou, Siberski, and Nejdl [PSN10] estimate the sizes of unions and intersections from filters. The region-weighted forms used here follow the composition manuscript Towell [Tow26f].

BF What this chapter adds. Two things, both modest. First, the perfect-filter-line classification of Sections 6.3 and 6.4: each one-sided axis is closed under union and intersection, complement exchanges the axes, and same-axis difference leaves the line. These facts are elementary once the region rates are written down; their use is organizational, for chapters 14 and 15. Second, the explicit caveat that a composed set is higher order, so its summary pair is not a valid operand for further composition (Remark 6.1.1); exact propagation is block by block, and the honest two-number summary is a worst-case bound.

BF Left out. Symmetric difference and the propagation of uncertain rates as intervals, both treated in the composition manuscript; operands that share hash functions or storage, beyond Remark 6.1.2, which chapter 16 takes up as correlated noise; and estimation of the cardinality of a composed set from its operands.

Chapter 7
Classification Measures

7.1 The Setup Revisited

Section 1.4 posed the estimation problem and stopped short of the theory. An approximate set \(\ASet {A} : \Universe \to \Bernoulli {\BoolType }\) arrives with two unknown rates: \(\fprate \), the chance it reports \(\top \) on a \(\bot \)-input, and \(\fnrate \), the chance it reports \(\bot \) on a \(\top \)-input. Query it on \(n\) labelled inputs, split them into the \(n_0\) trials whose truth is \(\bot \) and the \(n_1 = n - n_0\) whose truth is \(\top \), count false positives and false negatives, and report \(\hat {\fprate } = \mathrm {FP}/n_0\), \(\hat {\fnrate } = \mathrm {FN}/n_1\). Chapter 1 stated the binomial form, the unbiasedness, and a Wald confidence interval, all without proof, and deferred four things: the estimator’s exact properties (bias, variance, the MLE identity, efficiency); proper confidence intervals to replace the Wald form; the cause of the sawtooth under-coverage its notebook displayed; and the derived measures, precision and the predictive values, that depend on the workload as the rates do not. This chapter supplies all four, in that order, and closes with a deployed Bloom filter run through the whole toolkit.

BF Notation. The rates of Section 1.2 carry their usual symbols: \(\fprate \) and \(\fnrate \) for the two error rates, \(\tprate = 1 - \fnrate \) for the true-positive rate (recall or sensitivity), and \(\tnrate = 1 - \fprate \) for the true-negative rate (specificity); the hatted forms are their empirical estimators. Sample sizes are \(n_0\) (truth \(\bot \)) and \(n_1\) (truth \(\top \)), so \(\hat {\fprate } = \mathrm {FP}/n_0\) and \(\hat {\fnrate } = \mathrm {FN}/n_1\), and \(z_{\alpha /2}\) is the upper-\(\alpha /2\) standard-Normal quantile. The workload prevalence \(\pi = \Prob {X = \top }\) is the fraction of \(\top \)-inputs in the inference sample; PPV, NPV, and \(\Fone \) depend on it and \(\fprate , \fnrate \) do not.

7.2 The Binomial Estimator: Properties

The estimator \(\hat {\fprate } = \mathrm {FP}/n_0\) that Section 1.4 reported without proof is unbiased for \(\fprate \), has variance \(\fprate (1 - \fprate )/n_0\), equals the maximum-likelihood estimator, and attains the Cramér–Rao lower bound. Five short propositions settle these claims. Of the \(n\) trials, the \(n_0\) with truth \(\bot \) are the only ones a false positive can come from; for each, let \(X_i = 1\) if the noisy bit reports \(\top \). Axiom 1 makes the \(X_i\) i.i.d. \(\mathrm {Bernoulli} (\fprate )\), so \(\mathrm {FP} = \sum _{i=1}^{n_0} X_i \sim \mathrm {Binomial}(n_0, \fprate )\) and \(\hat {\fprate } = \mathrm {FP}/n_0\). The arguments for \(\hat {\fnrate } = \mathrm {FN}/n_1\) are identical under \(\fprate \to \fnrate \), \(n_0 \to n_1\).

The bias is zero at every \(\fprate \) and every \(n_0 \ge 1\), not merely in the limit.

The standard error shrinks as \(1/\sqrt {n_0}\), and the numerator \(\fprate (1 - \fprate )\) peaks at \(\fprate = 1/2\) and vanishes at the boundaries: the rate is hardest to pin down when the classifier is maximally noisy, the fact that returns in Section 7.3 as the reason Wald intervals deform near \(0\) and \(1\).

The empirical estimator is what likelihood theory asks for, not merely a convenient choice. It also equals the first-moment method-of-moments estimator (equate \(\Expect {\mathrm {FP}} = n_0 \fprate \) to the observed count). The coincidence is special to one-parameter exponential families like the binomial, whose variance is fixed by the mean; in a two-parameter model such as \(\mathrm {Beta}(a, b)\) the first moment is one equation in two unknowns, the moment hierarchy must reach the second moment to identify both parameters, and maximum likelihood and the method of moments generally diverge.

No unbiased estimator of \(\fprate \) does better in mean-squared error within the binomial model: a candidate reporting a lower variance is biased, trading bias for variance. What remains is to turn the point estimate into a confidence statement that holds at every \(n_0\), not only asymptotically, which Section 7.3 takes up.

7.3 Confidence Intervals Properly

Section 7.2 fixed \(\hat {\fprate } = \mathrm {FP}/n_0\) as the unbiased, efficient MLE. A point estimate is not a confidence statement, and the Wald interval of Section 1.4 fails in exactly the regime a practitioner most needs it. This section diagnoses the failure and derives three alternatives.

It reads the central-limit Normal \(\hat {\fprate } \sim \mathrm {Normal}(\fprate , \fprate (1 - \fprate )/n_0)\) off the sampling distribution and plugs \(\hat {\fprate }\) in for \(\fprate \) in the variance. Correct as \(n_0 \to \infty \), it fails at finite \(n_0\) near a boundary for two reasons. Boundary: the symmetric radius can push the lower endpoint below \(0\) or the upper above \(1\), assigning probability to impossible rates and needing ad hoc clipping. At \(\hat {\fprate } = 0\) it collapses to the degenerate point \([0,0]\). Skew: \(\mathrm {Binomial}(n_0, \fprate )\) is symmetric only at \(\fprate = 1/2\); near a boundary the central limit theorem has not overcome the binomial skew, and the symmetric interval miscalibrates, dropping below nominal coverage (near \(0.88\) at \(n_0 = 100\), \(\fprate = 0.05\) rather than \(0.95\)). A third, finer effect is the sawtooth: coverage as a function of \(n_0\) oscillates jaggedly because the interval is continuous while \(\mathrm {FP}\) is integer-valued, so the cover/miss boundary shifts in discrete jumps. The sawtooth afflicts every interval on a discrete distribution; Wald is unusual only in how far its dips fall below nominal.

Wilson, Clopper–Pearson, Jeffreys

Wilson inverts the score statistic, which keeps the true \(\fprate \) in the variance rather than its estimate. The confidence set \(\{\fprate : |Z(\fprate )| \le z\}\) with \(Z(\fprate ) = (\hat {\fprate } - \fprate )/ \sqrt {\fprate (1 - \fprate )/n_0}\) and \(z = z_{\alpha /2}\) squares to the quadratic \[ \fprate ^2\Bigl (1 + \tfrac {z^2}{n_0}\Bigr ) - 2\fprate \Bigl (\hat {\fprate } + \tfrac {z^2}{2 n_0}\Bigr ) + \hat {\fprate }^2 \le 0, \] whose two roots are the interval endpoints.

The center is \(\hat {\fprate }\) shrunk toward \(1/2\), and the radius carries a correction \(z^2/(4 n_0^2)\) that keeps the interval non-degenerate at \(\hat {\fprate } \in \{0, 1\}\): at \(\hat {\fprate } = 0\) it gives \([0, z^2/(n_0 + z^2)]\), admitting small positive rates. Both endpoints lie in \([0,1]\) by construction, killing the boundary problem, and the shift toward \(1/2\) corrects the direction the symmetric Wald interval got wrong.

Clopper–Pearson abandons the Normal approximation and inverts the exact binomial test: \(\fprate \) is included iff the observed \(\mathrm {FP}\) is consistent with \(\mathrm {Binomial}(n_0, \fprate )\) at level \(\alpha \). The endpoints are Beta quantiles, via the binomial–Beta identity \(\Prob {\mathrm {Binomial}(n_0, \fprate ) \ge k} = \Prob {\mathrm {Beta}(k, n_0 - k + 1) \le \fprate }\).

Called exact for using the exact binomial CDF, it guarantees coverage at least \(1 - \alpha \) but, because the binomial is discrete, over-covers at most \(\fprate \), sometimes substantially. Conservatism is the price of the guarantee.

Jeffreys reports a posterior credible interval under the invariant prior of Jeffreys [Jef46], \(\pi _J(\fprate ) \propto \sqrt {I(\fprate )} \propto \fprate ^{-1/2}(1 - \fprate )^{-1/2}\), the \(\mathrm {Beta}(1/2, 1/2)\) kernel; by conjugacy the posterior is \(\mathrm {Beta}(\mathrm {FP} + 1/2, n_0 - \mathrm {FP} + 1/2)\).

It resembles Clopper–Pearson with a half-integer shift but treats the endpoints symmetrically and is non-degenerate at the boundaries; its frequentist coverage is close to nominal at small \(n_0\) without Clopper–Pearson’s heavy conservatism.

Comparison and recommendation

The Wilson interval is this book’s default and is what Section 7.4 feeds into the delta method: it corrects the Wald boundary failure without Clopper–Pearson’s conservatism and needs only a Normal-quantile table. Jeffreys is the Bayesian alternative with excellent frequentist coverage; Clopper–Pearson is the conservative choice when an absolute coverage floor is required; Wald is acceptable only at large \(n_0\) away from the boundaries, where all four agree. Section 7.4 now takes up the derived measures and the delta-method translation of these intervals.

7.4 Derived Measures

The fourth deferred item is the derived measures. Precision, recall, \(\Fone \), and the two predictive values appear throughout binary classification and information retrieval, and Section 1.2 noted that precision depends on a quantity the noisy bit does not control: the workload’s prevalence of \(\top \). This section makes that precise, writing each measure as a function of \((\fprate , \fnrate , \pi )\) and propagating the binomial confidence intervals through the delta method.

Definitions

Apply a noisy bit with rates \(\fprate , \fnrate \) to a workload of \(n\) inputs whose truth is i.i.d. \(\mathrm {Bernoulli}(\pi )\) with \(\pi = \Prob {X = \top }\). The expected contingency-table counts are \[ \begin {aligned} \Expect {\TP } &= n\pi (1 - \fnrate ), & \qquad \Expect {\FN } &= n\pi \fnrate , \\ \Expect {\FP } &= n(1 - \pi )\fprate , & \qquad \Expect {\TN } &= n(1 - \pi )(1 - \fprate ), \end {aligned} \] the true-\(\top \) counts summing to \(n\pi \), the true-\(\bot \) counts to \(n(1 - \pi )\). Recall is the row-conditional rate \(\Recall = \tprate = 1 - \fnrate \), with \(\pi \) cancelling: recall is a property of the noisy bit alone. The predictive values and \(\Fone \) are column quantities, and \(\pi \) does not cancel.

\(\Fone \) weights precision and recall equally; the weighted family \(F_\beta = (1 + \beta ^2)\Precision \Recall /(\beta ^2\Precision + \Recall )\) tilts toward recall for \(\beta > 1\) and precision for \(\beta < 1\), recovering \(\Fone \) at \(\beta = 1\). Replacing the parameters by their estimates turns each formula into a point estimator.

Prevalence dependence

Read PPV as a function of \(\pi \) at fixed rates. At \(\pi = 0\) the numerator vanishes, so \(\PPV (0) = 0\) when \(\fprate > 0\): a workload with no genuine positives reports only false positives. At \(\pi = 1\) the denominator collapses to the numerator and \(\PPV (1) = 1\). Between, PPV rises monotonically, steeply near \(\pi = 0\) when \(\fprate \) is appreciable; NPV runs the mirror story, \(\NPV (0) = 1\), \(\NPV (1) = 0\). The framing is the point. The pair \((\fprate , \fnrate )\) is intrinsic to the classifier; precision is a joint property of classifier and workload, and a vendor who reports only precision is reporting what their evaluation workload happened to look like; reading a high \(\tprate \) and \(\tnrate \) as a high PPV on a low-prevalence workload is the base-rate fallacy. The book uses \((\fprate , \fnrate )\) as the primary description for exactly this reason.

Not every scalar summary depends on the workload. Youden’s index [You50], \[ J \;=\; \tprate + \tnrate - 1 \;=\; 1 - \fprate - \fnrate , \] uses only the two intrinsic rates. It is zero exactly when the report is independent of the truth (\(1 - \fnrate = \fprate \)), and it is the height of the operating point above the chance diagonal of the receiver-operating-characteristic plane. Accuracy, \(\pi (1 - \fnrate ) + (1 - \pi )(1 - \fprate )\), is by contrast a workload quantity, a prevalence-weighted average of \(\tprate \) and \(\tnrate \).

Estimation and the delta method

The estimator \(\widehat {\PPV } = \TP /(\TP + \FP )\) is a ratio of two independent binomial counts. Conditioning on the row totals \(n_\top \) and \(n_\bot \), \[ \TP \sim \mathrm {Binomial}(n_\top , \tprate ), \qquad \FP \sim \mathrm {Binomial}(n_\bot , \fprate ), \] independent by Axiom 1. A ratio’s expectation is not the ratio of expectations, so \(\widehat {\PPV }\) is biased (small when both means are large, and consistent as \(n \to \infty \)), and its variance is read off by the delta method rather than directly.

This is the first-order Taylor expansion \(g(\hat \theta ) \approx g(\theta ) + g'(\theta )(\hat \theta - \theta )\) with Slutsky’s lemma; a proof is in Casella and Berger [CB02, Ch. 5]. The multivariate form replaces \(g'(\theta )\) by the gradient and \(\sigma ^2\) by a covariance matrix, giving the quadratic form \(\nabla g^\top \Sigma \, \nabla g\).

A Wald-style PPV interval is \(\widehat {\PPV } \pm z_{\alpha /2} \sqrt {\widehat {\Var [\widehat {\PPV }]}}\), which inherits the boundary and skew failures of the Wald rate interval; the remedy is a Wilson-style score interval on the column-conditional binomial \(\TP \mid \TP + \FP \sim \mathrm {Binomial}(\TP + \FP , \PPV )\), which stays inside \([0,1]\) by construction, or a bootstrap over the contingency table.

\(\Fone \) via the delta method

Writing \(\Fone = 2\TP /(\TP + \FP + n_\top )\) (using \(\FN = n_\top - \TP \)) and applying Theorem 7.4.1 to \(h(t, f) = 2t/(t + f + n_\top )\) gives \[ \Var [\Fone ] \approx \frac {4(\bar f + n_\top )^2\sigma _t^2 + 4\bar t^{\,2}\sigma _f^2}{(\bar t + \bar f + n_\top )^4}, \] differing from the PPV variance only by the \(n_\top \) added to the \(\TP \)-gradient component.

Example 7.1 (\(\Fone \) with delta-method CI) Take \(\fprate = 0.05\), \(\fnrate = 0.10\), \(\pi = 0.1\), \(n = 1000\), so \(n_\top = 100\), \(n_\bot = 900\). Then \(\Recall = 0.90\) and \[ \Precision = \frac {0.1 \cdot 0.9}{0.1 \cdot 0.9 + 0.9 \cdot 0.05} = \frac {0.09}{0.135} = \frac {2}{3}, \] and \[ \Fone = \frac {2 \cdot (2/3) \cdot 0.9}{2/3 + 0.9} \approx 0.766. \] With \(\bar t = 90\), \(\bar f = 45\), \(\sigma _t^2 = 100 \cdot 0.9 \cdot 0.1 = 9\), \(\sigma _f^2 = 900 \cdot 0.05 \cdot 0.95 = 42.75\), and \(s = 90 + 45 + 100 = 235\), \[ \Var [\Fone ] \approx \frac {4 \cdot 145^2 \cdot 9 + 4 \cdot 90^2 \cdot 42.75}{235^4} = \frac {756900 + 1385100}{3{,}049{,}800{,}625} \approx 7.02 \times 10^{-4}, \] a standard deviation of \(0.0265\), so the \(95\%\) interval is \(0.766 \pm 1.96 \cdot 0.0265 \approx [0.714, 0.818]\). The interval is narrower than the analogous PPV interval because the \(n_\top \) in the \(\Fone \) denominator damps the response to a change in either count.

Bloom filter PPV under workload

The prevalence dependence is sharp enough to drive a concrete case. Take a Bloom filter (Definition 5.2.1, Section 5.2) with \(\fnrate = 0\) and \(\fprate = 0.01\), deployed on two workloads. At prevalence \(\pi = 0.01\), \[ \PPV = \frac {0.01 \cdot 1}{0.01 \cdot 1 + 0.99 \cdot 0.01} = \frac {0.01}{0.0199} \approx 0.503, \] so half of all positive reports are false: the filter is right on every \(\top \) (by \(\fnrate = 0\)) but wrong on \(1\%\) of the much larger \(\bot \) population, and the two roughly match at the column total. At \(\pi = 0.5\), \[ \PPV = \frac {0.5 \cdot 1}{0.5 \cdot 1 + 0.5 \cdot 0.01} = \frac {0.5}{0.505} \approx 0.990. \] Same filter, same \((\fprate , \fnrate )\), PPV moving by nearly a factor of two across workloads. “The Bloom filter has precision \(0.99\)” is uninformative without the workload; the \((\fprate , \fnrate )\) claim is workload-free. Section 7.5 runs the inverse problem, recovering a PPV interval from an observed false-positive count at known prevalence.

7.5 Worked Application: Bloom Filter Diagnostics

One deployment problem exercises the whole toolkit: a spam filter implemented as a Bloom filter, with a question, is the filter precise enough on this workload, and if not, what to change.

Scenario and observation

The filter (Definition 5.2.1, Section 5.2) has \(m = 50000\) bits, \(k = 7\) hashes, and \(n = 5000\) items. The ratio \(m/n = 10\) with \(k = 7\) puts it in the optimal regime of Theorem 5.2.1, and (5.1) at load \(kn/m = 0.7\) predicts \[ \fprate \approx (1 - e^{-0.7})^7 \approx 0.0082, \] with \(\fnrate = 0\) by construction. A workload of \(N = 10000\) messages arrives, \(n_\top = 1000\) spam and \(n_\bot = 9000\) not; the filter reports \(\top \) on all spam and on \(\mathrm {FP} = 92\) of the non-spam, so \[ \hat {\fprate } = \frac {92}{9000} \approx 0.01022, \] about \(25\%\) above the design value. The Wilson \(95\%\) interval (Definition 7.3.2) at \(z = 1.96\) has center \((0.01022 + 1.96^2/18000)/(1 + 1.96^2/9000) \approx 0.01043\) and half-width \[ \frac {1.96}{1.000427}\sqrt {\frac {0.01022 \cdot 0.98978}{9000} + \frac {1.96^2}{4 \cdot 9000^2}} \approx 0.00209, \] giving \([0.00834, 0.01252]\). The design \(\fprate = 0.0082\) sits just below the lower endpoint: at \(n_\bot = 9000\) the interval is narrow enough to detect the \(25\%\) deviation, plausibly from hash collisions that mildly violate the uniform-hash assumption or an item count above the design \(n\). The deployment \(\fprate \) is an empirical quantity to measure, not a design parameter to trust; we carry \(\hat {\fprate } = 0.0102\) forward.

PPV and its interval

With \(\hat \pi = n_\top /N = 0.1\), Definition 7.4.1 gives \[ \widehat {\PPV } = \frac {0.1 \cdot 1}{0.1 \cdot 1 + 0.9 \cdot 0.01022} = \frac {0.1}{0.10920} \approx 0.9158, \] so about \(92\%\) of flagged messages are spam. The delta-method variance (Proposition 7.4.1) simplifies because \(\fnrate = 0\) kills the \(\sigma _t^2\) term: with \(\bar t = 1000\), \(\bar f = 92\), \(\sigma _f^2 = 9000 \cdot 0.01022 \cdot 0.98978 \approx 91.06\), \[ \Var [\widehat {\PPV }] \approx \frac {1000^2 \cdot 91.06}{1092^4} \approx 6.40 \times 10^{-5}, \] a standard error of \(0.0080\), so the \(95\%\) interval is \(0.9158 \pm 1.96 \cdot 0.0080 \approx [0.900, 0.932]\). The boundary concern of Section 7.3 does not bite (\(\widehat {\PPV }\) is far from \(0\) and \(1\), the conditional samples are large). All uncertainty here comes from the false-positive count; were \(\fnrate > 0\), recall would be random and both variance terms would contribute.

The diagnostic

Suppose the requirement is \(\PPV > 0.95\). The estimate \(0.9158\) falls short and the interval \([0.900, 0.932]\) excludes \(0.95\): a fix is needed, and the interval, not the point, is what establishes it. A point estimate of \(0.9158\) alone would have suggested the target was nearly met; the interval rules out \(0.95\) at \(95\%\) confidence. Definition 7.4.1 has three knobs, and \(\fnrate = 0\) already, leaving two routes.

Route A, re-tune. Lower \(\fprate \). Solving \(0.95 = 0.1/(0.1 + 0.9 \fprate )\) gives \(\fprate \approx 0.00585\); at \(\fprate = 0.005\) the PPV is \(0.1/(0.1 + 0.9 \cdot 0.005) \approx 0.957\), above target. Reaching \(0.005\) from \(0.0102\) by (5.1) costs about \(16\%\) more bits at fixed \(k\), a one-time hardware decision that buys an \(\fprate \) surviving the worst-case workload.

Route B, wait. Keep the filter and bound the acceptable workload. Solving \(0.95 = \pi /(\pi + (1 - \pi ) \cdot 0.01022)\) gives \(\pi \approx 0.163\); at \(\pi = 0.2\) the PPV is \(\approx 0.961\). Doing nothing structural is acceptable once the spam rate rises above about one in six, but it trades capital cost for the assumption that the workload mix will shift, which the deployment does not control.

The choice is operational, not statistical. Both routes used the same three tools, (5.1) for the design rate, the Wilson interval for its empirical estimate, and Definition 7.4.1 for the translation into precision at prevalence. What the chapter added beyond the chapter-5 design formulas is the honest accounting of uncertainty that turns “nearly met” into “ruled out.”

7.6 Notes and Lineage

BF Lineage. Every tool in the chapter is standard. The predictive values and their dependence on prevalence are the core of diagnostic-test epidemiology, where sensitivity and specificity play the roles of \(\tprate \) and \(\tnrate \), and the prevalence-free index \(J = 1 - \fprate - \fnrate \) is due to Youden [You50]; Goutte and Gaussier [GG05] give a probabilistic treatment of precision, recall, and \(\Fone \) with posterior distributions. The score interval is due to Wilson [Wil27], the exact test-inversion interval to Clopper and Pearson [CP34], and the invariant prior behind the Jeffreys interval to Jeffreys [Jef46]. The failure of the Wald interval and the coverage sawtooth are documented by Agresti and Coull [AC98] and Brown, Cai, and DasGupta [BCD01], whose recommendations the chapter follows; the delta method is textbook material [CB02, Ch. 5]. The chapter’s derivations follow the classification-measures manuscript Towell [Tow26c].

BF What this chapter adds. No new statistics. The chapter places these tools inside the channel model, separating the two intrinsic rates of an approximate set from the workload-dependent measures computed from them, and works a deployed Bloom filter through the full chain from design rate to interval to decision (Section 7.5).

BF Left out. Receiver-operating-characteristic analysis and threshold selection; multi-class confusion matrices, which chapter 9 treats as \(n\)-ary channels; and a caveat on sampling units. The binomial model treats the queries of one built structure as independent trials, which they are only conditionally on that build. An interval computed from one filter therefore estimates that filter’s rate, not the mean rate of the construction; estimating the latter requires independent builds, a point chapter 16 develops under correlated noise.

Chapter 8
Information-Theoretic Bounds

8.1 Shannon Entropy Refresher

The chapter’s two results, the BSC capacity (Section 8.2) and the entropy floor for approximate sets (Section 8.3), rest on a small slice of Shannon’s information theory: the entropy of a random variable, its binary special case, and mutual information. We collect them and fix notation; Cover and Thomas [CT06] is the depth reference.

BF Entropy. For a discrete \(X\) on a finite alphabet \(\mathcal {X}\), the Shannon entropy is \begin {equation} \label {eq:entropy} H(X) \;=\; -\sum _{x \in \mathcal {X}} \Prob {X = x}\,\log _2 \Prob {X = x}, \end {equation} with \(0 \log _2 0 = 0\), measured in bits. Operationally \(H(X)\) is the average number of yes/no questions an optimal binary search needs to identify a draw: \(0\) for a constant, \(\log _2|\mathcal {X}|\) for the uniform distribution. By Shannon’s source-coding theorem it is also the tight lower bound on the expected length of any prefix-free code for \(X\). Section 8.3 uses the counting form of the same idea: singling out one of \(K\) equally possible objects takes \(\log _2 K\) bits.

BF Binary entropy. For \(X \sim \mathrm {Bernoulli}(p)\), (8.1) collapses to the binary entropy function \begin {equation} \label {eq:binary-entropy} H(p) \;=\; -p \log _2 p - (1 - p)\log _2(1 - p), \qquad p \in [0, 1], \end {equation} overloading \(H\) (a random variable in \(H(X)\), a scalar in \(H(p)\); context disambiguates). Three properties recur in this chapter and chapter 14: \(H\) is strictly concave (\(H''(p) = -1/[p(1-p)\ln 2] < 0\)); it peaks at \(H(1/2) = 1\) (a fair coin is the most uncertain bit); and \(H(0) = H(1) = 0\). It is symmetric, \(H(p) = H(1-p)\), with \(H(0.01) \approx 0.081\), \(H(0.1) \approx 0.469\). Distinct from \(H(\fprate )\) is the self-information (surprisal) \(-\log _2 \fprate \) of a single event of probability \(\fprate \): the floor of Section 8.3 is \(-\log _2 \fprate \) bits per element, the cost of one such event, not the indicator entropy \(H(\fprate ) \approx -\fprate \log _2 \fprate \) that weights it by \(\fprate \).

BF Mutual information and capacity. For jointly distributed \(X, Y\), the mutual information \begin {equation} \label {eq:mutual-info} I(X; Y) \;=\; H(X) - H(X \mid Y) \;=\; H(Y) - H(Y \mid X) \end {equation} measures how much observing one reduces uncertainty about the other, with \(H(X \mid Y) = \sum _y \Prob {Y = y}\,H(X \mid Y = y)\) the conditional entropy. Equivalently \begin {equation} \label {eq:mutual-info-joint} I(X; Y) \;=\; \sum _{x, y} \Prob {X = x, Y = y} \log _2 \frac {\Prob {X = x, Y = y}}{\Prob {X = x}\Prob {Y = y}}, \end {equation} a Kullback–Leibler divergence between the joint and the product of marginals, hence nonnegative and zero iff \(X, Y\) are independent. For a channel with fixed \(\Prob {Y \mid X}\), \(I(X; Y)\) depends on the input distribution, and the capacity is the maximum, \begin {equation} \label {eq:capacity-def} C \;=\; \max _{\Prob {X}} I(X; Y), \end {equation} the most information one channel use can convey. Section 8.2 evaluates it for the BSC of Section 2.1 and gets \(C = 1 - H(\fprate )\); Section 8.3 applies the counting reading to approximate-set representations and gets the \(-\log _2 \fprate \) floor; Section 8.4 measures the standard Bloom filter’s \(\log _2 e \approx 1.443\) overhead above it.

8.2 Channel Capacity in the BSC

The binary symmetric channel of Section 2.1 reproduces its Boolean input with probability \(1 - \fprate \) and flips it with probability \(\fprate \) regardless of the value sent, the channel matrix being \(Q = \left (\begin {smallmatrix} 1 - \fprate & \fprate \\ \fprate & 1 - \fprate \end {smallmatrix}\right )\) (Definition 2.1.1). Its capacity is the quantitative information-theoretic summary chapter 2 left implicit.

Once \(\fprate \) is fixed the channel admits at most \(C\) bits per use, and by Shannon’s noisy-channel coding theorem this ceiling is asymptotically achievable: for any rate \(R < C\) there are long block codes with decoding error below any \(\varepsilon \). The closed form has a sharp shape. At \(\fprate = 0\) the channel is noiseless, \(H(0) = 0\) and \(C = 1\). At \(\fprate = 1/2\) the output is a fair coin independent of the input, \(H(1/2) = 1\) and \(C = 0\). At \(\fprate = 1\) the channel always flips, which a receiver undoes by relabeling \(\top \leftrightarrow \bot \), so \(H(1) = 0\) and \(C = 1\) again; \(C(\fprate )\) is symmetric about \(\fprate = 1/2\). The curve is steep near \(0\) (the first crossover noise costs little: \(C(0.01) \approx 0.919\)) and shallow near \(1/2\) (the last noise costs almost everything: \(C(0.1) \approx 0.531\), \(C(0.4) \approx 0.029\)).

The rest of the chapter asks a different question with the same currency: how many bits must represent an approximate set at target \(\fprate \)? Capacity bounds what a noisy channel can carry per use; the floor of Section 8.3 bounds what a representation must store, by counting how many target sets each stored string can stand for. The answer, \(-\log _2 \fprate \) bits per element for any FNR-zero representation, is a self-information rather than an entropy \(H(\fprate )\), and the counting argument that gives it needs no channel coding.

8.3 The Entropy Floor for Approximate Sets

The bit-budget comparison of Section 5.4 placed the Bloom filter at roughly \(9.6\) bits per item at \(\fprate = 0.01\) against a floor near \(6.64\). This section states that floor as a theorem, the classical counting bound of Carter et al. [Car+78]: any approximate set with \(\fnrate = 0\) and false-positive rate at most \(\fprate \), in any representation whatever, must store about \(\log _2(1/\fprate )\) bits per element. It then gives the two-sided form, due to Pagh and Rodler [PR01], and the constructive picture.

Setup

Fix a universe \(\Universe \) with \(|\Universe | = N\) and a target \(A \subseteq \Universe \) with \(|A| = n\). An approximate-set representation is a pair \((R, \mathrm {query})\) with \(R \in \{0,1\}^L\) a bit string and \(\mathrm {query}(R, \cdot ) : \Universe \to \{\top , \bot \}\) a deterministic membership algorithm; \(L\) is the cost to bound. Following Definition 5.4.1 we restrict to the FNR-zero family containing the Bloom filter: \(\mathrm {query}(R, x) = \top \) for every \(x \in A\) (no inserted element is missed), and at most a \(\fprate \) fraction of the \(N - n\) non-members is accepted. The accepted set \(A_R = \{x : \mathrm {query}(R, x) = \top \}\) then satisfies \(A \subseteq A_R\) and \(|A_R \setminus A| \le \fprate (N - n)\), so \begin {equation} \label {eq:accepted-size} |A_R| \;\le \; M \;:=\; n + \fprate (N - n). \end {equation} The verdict on \(x\) depends on \(R\) only through which side of \(A_R\) the query lands, so \(A_R\) is the representation-independent object that does the work.

The theorem

The condition \(n/(\fprate N) \to 0\) is the deployment regime: \(\fprate \) fixed and the target embedded in a much larger universe. There the floor is sharp; when \(\Universe \) is barely larger than \(A\), the \(n/N\) term matters and the per-element reading degrades, and at \(\fprate = 1\) the bound is vacuous, met by the trivial always-accept structure that stores nothing. The standard formulation of the bound also allows the accepted superset to be random [PPR05]; the deterministic statement above is its core.

Form of the bound

The bound has no parameters to tune: the per-element cost is a logarithm of the inverse error rate, independent of how the representation is built.

A million-element FNR-zero set at one-in-a-billion false-positive rate cannot fit in less than about \(3.74\) megabytes, whatever the encoding. The cost grows additively, by \(\log _2 10 \approx 3.32\) bits per element for each order of magnitude in \(1/\fprate \), not geometrically. The universe size \(N\) does not enter the leading term: one might have expected the \(\log _2\binom {N}{n}\) cost of identifying \(A\) exactly, but the FPR budget buys back precisely that universe-dependent excess, leaving the residual \(n\log _2(1/\fprate )\).

The two-sided case

The proof of Theorem 8.3.1 uses FNR-zero to force \(A \subseteq A_R\), and that containment is what makes \(\binom {\lfloor M \rfloor }{n}\) the right count. If members may be rejected, \(A\) need not sit inside \(A_R\) and the count changes. The answer is nevertheless known to leading order.

This is Proposition 1 of Pagh and Rodler [PR01], stated there for the lossy dictionaries they introduce; we do not reprove it. The bound is matched to leading order in \(\log (1/\fprate )\) by the obvious construction: keep any \((1 - \gamma )n\) members in a space-optimal FNR-zero structure and drop the rest. In the per-element model of this book a false-negative rate \(\fnrate \) rejects each member independently, so the number rejected concentrates near \(\fnrate n\), and up to that concentration step the proposition applies with \(\gamma \approx \fnrate \). Tolerating false negatives therefore buys space only linearly: the saving is about \(\fnrate \log _2(1/\fprate )\) bits per element, the cost of the members one chooses not to store. To leading order, no two-sided representation blurs the identity of \(A\) more cheaply than by forgetting part of it.

The constructive question

Theorem 8.3.1 forbids: no FNR-zero scheme beats \(-\log _2 \fprate \) bits per element. That the floor can be met, up to lower-order terms, is also established. Pagh, Pagh, and Rao [PPR05] gave an optimal replacement for the Bloom filter, and Dietzfelbinger and Pagh [DP08] reduced approximate membership to retrieval, storing a \(\log _2(1/\fprate )\)-bit fingerprint function in near-optimal space. Practical descendants come close: xor filters are within about \(23\%\) of the bound and binary fuse filters within about \(13\%\) [GL20; GL22], and ribbon filters [Die+26] closer still. For structures that must accept insertions one at a time, as the Bloom filter does, Lovett and Porat [LP10] prove a lower bound strictly above the static floor at constant \(\fprate \). The standard Bloom filter pays a factor of about \(1.44\) (Section 8.4). Chapter 14 (Chapter 14) presents the static perfect hash filter, a minimal perfect hash plus fingerprints at \(\log _2(1/\fprate ) + c\) bits per element with \(c \ge \log _2 e\), and a seed-search construction, the Bernoulli Hash Function, which meets the floor in space but needs on the order of \(\fprate ^{-n}\) trials to build; it is a reference point rather than a practical structure.

8.4 The Bloom Filter’s Gap

The standard Bloom filter is FNR-zero (Definition 5.2.1), so Theorem 8.3.1 governs it. How far above the floor does it sit? The answer is a single constant, \(\log _2 e \approx 1.443\), the same at every \(\fprate \).

Bits per element

By Theorem 5.2.1 the optimal Bloom filter is half-occupied with \begin {equation} \label {eq:bloom-optimal-fpr-recall} \fprate ^* \;=\; 2^{-(m/n)\ln 2}. \end {equation} Inverting at \(\fprate ^* = \fprate \), \(\log _2 \fprate = -(m/n)\ln 2\), so \begin {equation} \label {eq:bloom-bits-per-element} \frac {m}{n} \;=\; \frac {\log _2(1/\fprate )}{\ln 2} \;=\; \log _2(1/\fprate )\cdot \log _2 e, \end {equation} using \(1/\ln 2 = \log _2 e\). The Bloom budget is thus a constant multiple \(\log _2 e\) of the floor cost \(\log _2(1/\fprate )\) of Theorem 8.3.1, so the ratio is \(\log _2 e = 1/\ln 2 \approx 1.4427\), independent of \(\fprate \).

The factor is classical: it follows from the optimal-\(k\) analysis of the filter and the floor of Carter et al. [Car+78], and it is the standard benchmark in the literature [BM04].

The constant is the chapter-5 half-occupancy result read dually: \(\ln 2 = 1/\log _2 e\) is the optimal load that maximizes per-bit entropy, and the same \(\log _2 e\) is the gap to the floor. Half-occupancy is information-theoretically the wrong target, but it is the best a representation built from \(k\) independent hashes and a flat bit array can do.

The gap in absolute terms

A constant factor on a function that grows like \(\log _2(1/\fprate )\) produces an absolute gap that grows with it:

At one-percent FPR the filter pays about three extra bits per element; at one-in-a-million, about nine. For a million-element set at \(\fprate = 10^{-6}\) the gap is \(8.83\) megabits, about \(1.10\) megabytes, that the standard construction cannot recover. A constant factor on an already-small quantity is acceptable in many settings, and Bloom filters dominate the ecosystem for good reason; the point of Proposition 8.4.1 is to quantify the slack, not to condemn the construction, and to ask which design choice forces it. Figure 8.1 draws the comparison over the whole range of rates.

Why the gap is structural

The Bloom filter pays its overhead because of how it is built: \(k\) independent hashes write into a single flat bit array, and a query accepts iff all \(k\) read bits are \(1\). The optimal-\(k\) analysis sets the per-bit fill to \(p = 1/2\), where the array carries one full bit of Shannon entropy per array bit and the encoding is maximally dense at the bit level. But the floor is set by the membership problem, not by the encoding’s internal density. The Bloom filter maximizes per-bit entropy and takes the \(k\)th power of it, getting a clean optimum at half-occupancy while spending bits on the array’s internal balance rather than on the discrimination it was meant to perform.

The retrieval literature does exactly that. A perfect hash gives each element a location, and a fingerprint of \(\log _2(1/\fprate )\) bits is stored there, so fingerprint coincidences are the only source of false positives. The perfect hash costs a constant \(c \ge \log _2 e\) bits per element, so the multiplicative \(\log _2 e\) factor disappears and the relative overhead vanishes as \(\fprate \to 0\); at a fixed rate the construction beats an optimal Bloom filter only when \((\log _2 e - 1)\log _2(1/\fprate ) > c\). Dietzfelbinger and Pagh [DP08] describe this construction, noting that its origin is unknown to them, and reduce approximate membership to retrieval in general; xor, binary fuse, and ribbon filters are its engineered descendants (Section 8.3). Chapter 14 presents it as the perfect hash filter (Definition 14.3.1), alongside the Bernoulli Hash Function of Definition 14.2.1, a single-salt search that meets the floor in space but needs on the order of \(\fprate ^{-n}\) trials to build. The gap is not a deficiency of analysis but a consequence of a specific structural choice, and a different choice closes it.

8.5 What Comes Next

Part II is complete. From the approximate set as a noisy bit per universe element (Chapter 5) it developed the algebra of unions, intersections, and complements (Chapter 6), the estimation and diagnostic discipline for the two rates (Chapter 7), and the information-theoretic floor that pins the per-element space cost of any FNR-zero representation (this chapter). The atom is now fully described: its rates are estimable, its compositions predictable, its space cost bounded below by \(-\log _2 \fprate \) (Theorem 8.3.1), and the standard Bloom filter sits a known \(\log _2 e\) above that floor (Section 8.4).

Part III generalizes from sets to maps: Chapter 9 lifts \(\Bernoulli {\BoolType }\) to \(\Bernoulli {T}\) for arbitrary algebraic types, where an approximate map into a \(k\)-element output is a \(k \times k\) channel rather than a single \((\fprate , \fnrate )\) pair, and chapters 10 through 13 develop sketches, randomized algorithms, approximate relations, and the formal type framework. Part IV builds the constructive payoff: Chapter 14 abandons the flat bit array for perfect hashing, presenting the perfect hash filter and the Bernoulli Hash Function, which meets the floor in space at an exponential construction cost, and Chapter 15 builds the FPR-zero counterpart on the other one-sided axis. Part V draws the boundary: Chapter 16 catalogues the systems that are not Bernoulli, where errors correlate across inputs, rates drift, or adversarial queries defeat uniform sampling, and the closing chapters turn the model’s controlled error into a cryptographic resource. The thread is one throughout: an atom of controlled error, its algebra, its floor, its optimal constructions, and the boundary that fences off what the atom cannot describe.

8.6 Notes and Lineage

BF Lineage. The floor of Theorem 8.3.1 is the counting bound of Carter et al. [Car+78], who introduced approximate membership testers and proved that about \(n\log _2(1/\fprate )\) bits are necessary. Its two-sided form, Proposition 8.3.1, is Proposition 1 of Pagh and Rodler [PR01]; the false-negative case is therefore not open. That the floor is attainable up to lower-order terms is due to Pagh, Pagh, and Rao [PPR05] and, through the reduction to retrieval, to Dietzfelbinger and Pagh [DP08]; Lovett and Porat [LP10] show that structures supporting insertions cannot reach it at constant \(\fprate \). The Bloom filter’s factor of \(\log _2 e\) is the classical benchmark [BM04], and the compressed filter of Remark 8.4.1 is due to Mitzenmacher [Mit02]. Entropy, mutual information, and the BSC capacity are from Shannon [Sha48]; Cover and Thomas [CT06] is the reference treatment.

BF What this chapter adds. Presentation. The pigeonhole count is written in the exact form \(L \ge n\log _2(1/(\fprate + n/N))\); the floor is read as a self-information \(-\log _2\fprate \) per element rather than an entropy \(H(\fprate )\); and the two-sided bound is interpreted in the book’s per-element model, where it says that false negatives buy space only linearly. The computations in the program’s notation, including the joint entropy of the error counts, are in Towell [Tow26g]; its earlier statement that the two-sided case is open is superseded by Proposition 8.3.1.

BF Left out. Lower-order terms and the precise constants of near-optimal constructions; dynamic constructions and their time-space tradeoffs; the joint entropy of the false-positive and false-negative counts; and adaptive query sequences, for which the guarantees change [Ben+18] and which chapter 16 takes up.

Part III
From Sets to Maps

Chapter 9
Approximate Functions and n-ary Channels

9.1 From Approximate Sets to Approximate Functions

Part II read a classical set as its membership predicate \(\SetIndicator {A} : \Universe \to \{\top , \bot \}\) and made the predicate noisy: each evaluation \(\ASet {A}(x)\) is a \(\Bernoulli {\BoolType }\) instance, and the chapter-4 axioms collapse the per-element rates to a single pair \((\fprate , \fnrate )\). The framing pinned the codomain to \(\{\top , \bot \}\). Nothing in it requires two values, only that each evaluation draws from a distribution on the codomain fixed by the input’s block. Replacing \(\{\top , \bot \}\) with any type \(T\) gives the approximate function \[ \tilde {f} : \Universe \longrightarrow T, \] whose evaluation \(\tilde {f}(x)\) is an instance of the lifted type \(\Bernoulli {T}\), the \(T\)-valued analogue of the noisy bit. For \(T = \BoolType \) this is exactly the membership predicate of an approximate set, so Part II is the special case; for \(T = \{1, \ldots , K\}\) it is a \(K\)-way classifier; for \(T = \mathbb {N}\) it is a counter or frequency estimator. The two-rate description gives way to a full per-block output distribution, and the \(2 \times 2\) channel matrix of Part II becomes a \(K \times L\) matrix with \(K\) input blocks, \(L = |T|\) output values, rows summing to one, and \((i,j)\)-entry the probability that block \(i\) produces output \(j\).

BF Two examples. A 3-class classifier on \(T = \{\text {cat}, \text {dog}, \text {bird}\}\) has its input domain split into three blocks by true class, with per-block behavior a \(3 \times 3\) confusion matrix: diagonal entries the per-class accuracies, off-diagonals the confusion rates, six free parameters once rows are normalized. A counter on \(T = \mathbb {N}\) has input blocks \(B_n\) indexed by true count \(n\), and row \(n\) is a distribution on \(\mathbb {N}\) concentrated near \(n\) with a tail width that records the counter’s precision; the matrix is countably infinite but each row is still a probability distribution. In both cases the structure is the same: a function from inputs to a typed codomain, with the per-query distribution fixed by the input’s block.

BF What transfers. The two chapter-4 axioms carry over unchanged, since both are independence claims indifferent to alphabet size: Axiom 1 keeps error events at distinct inputs independent, Axiom 2 keeps each per-block distribution stable. The parameter count grows from \(2\) to \(K(L-1)\) (each of \(K\) rows is a distribution on \(L\) outputs). Two structural results scale accordingly. The Kronecker factorization of Theorem 4.4.1 (Section 4.4), which factored the joint of \(m\) Boolean queries as a tensor product of \(2 \times 2\) matrices, holds with \(K \times L\) matrices in their place (Section 9.3). The BSC capacity \(C = 1 - H(\fprate )\) of Proposition 8.2.1 (Section 8.2) is specific to the binary case, but capacity as the maximum mutual information over input distributions generalizes, with closed forms for the symmetric sub-family and numerical recipes otherwise (Section 9.4). Section 9.2 gives the formal definition first.

9.2 N-ary Channels

Section 9.1 described an approximate function \(\tilde {f} : \Universe \to T\) whose per-block behavior is a distribution on \(T\). This section gives the formal object: a row-stochastic matrix, with the binary \(2\) of Part II replaced by the number of input blocks \(K\) and the output cardinality \(L = |T|\).

BF Model order versus row count. The integer \(K\) counts the rows of \(Q\), one per input block. The model order of Definition 2.4.1 is a different and generally smaller count: the number of blocks under the coarsest partition that merges inputs sharing a common equality-error rate \(\varepsilon (x) = \Prob {\tilde {f}(x) \neq x}\). The two coincide when every block has a distinct error rate, and the model order is strictly smaller when blocks share one. The distinction matters at the very first example. The binary symmetric channel writes as a \(2 \times 2\) matrix (\(K = 2\) rows) but is model order one: both inputs flip at the common rate \(\fprate \), so they form a single error-rate block (the first-order Bernoulli model of Section 2.4). The binary asymmetric channel is also \(2 \times 2\) but model order two, since its inputs carry different error rates \(\fnrate \neq \fprate \). Throughout this chapter \(K\) is the matrix row count; we name the model order only where the distinction is in play.

BF Notation: \(K\) uppercase, \(k\) lowercase. The uppercase \(K\) (input-block count) and \(L\) (output cardinality) are structural parameters of the channel. The lowercase \(k\) is reserved for the tuning knobs of Parts I and II, the Bloom-filter hash count (Section 5.2) and the Miller-Rabin round count (Section 3.5); the case distinction keeps the structural from the tunable. A Bloom filter has \(K = 2\) input blocks (members and non-members) and hash count \(k\); both blocks of a Bloom filter or a Miller-Rabin test carry different error rates (members never miss, non-members false-positive at \(\fprate \)), so these are genuinely model order two, while their lowercase \(k\) is a design choice.

Examples

BF Binary, symmetric and asymmetric. The BSC of Definition 2.1.1 is \(Q_{\mathrm {BSC}} = \left (\begin {smallmatrix} 1 - \fprate & \fprate \\ \fprate & 1 - \fprate \end {smallmatrix}\right )\): \(K = 2\) rows, \(L = 2\), model order one. Dropping the symmetry recovers the asymmetric channel of Definition 2.2.1, \(Q_{\mathrm {asym}} = \left (\begin {smallmatrix} 1 - \fnrate & \fnrate \\ \fprate & 1 - \fprate \end {smallmatrix}\right )\), with two independent rates and model order two; the Z-channel is the one-sided specialization \(\fnrate = 0\).

BF A 3-class classifier (\(K = L = 3\)). A confusion matrix is an n-ary channel. A classifier over \(\{\text {cat}, \text {dog}, \text {bird}\}\) has input blocks by true class and \[ Q \;=\; \begin {pmatrix} 0.90 & 0.05 & 0.05 \\ 0.05 & 0.90 & 0.05 \\ 0.05 & 0.05 & 0.90 \end {pmatrix}, \] rows summing to one, diagonal the per-class accuracies, off-diagonals the confusion rates. A generic 3-class classifier has three distinct per-class error profiles, model order three, and \(K(L - 1) = 3 \cdot 2 = 6\) free parameters. The displayed matrix is more constrained: all three classes share the equality-error rate \(0.10\) with an even off-diagonal split, so it is model order one, specified by the single number \(0.90\). The matrix dimension (\(3 \times 3\)) and the model order (\(1\)) are again different counts.

BF A counter (\(K = L = \infty \)). A frequency estimator on \(T = \mathbb {N}\) has input blocks \(B_n\) indexed by true count \(n\) and an infinite channel matrix whose row \(n\) is a distribution on \(\mathbb {N}\) concentrated near \(n\). CountMin gives a one-sided tail (it over-counts but never under-counts), HyperLogLog a relative-error tail; chapter 10 builds the constructions.

BF A physical instance: quantum readout. The n-ary channel has a direct physical realization in quantum computing. Measuring \(n\) qubits in the computational basis is a channel whose \(2^n \times 2^n\) row-stochastic matrix \(A_{tb} = \Prob {\text {observed bitstring } b \Given \text {basis state } t}\) is the assignment (or response) matrix that readout-error mitigation calibrates and inverts [Bra+21; Nat+21; MZO20]. When per-qubit readout errors are independent, \(A\) factors as a Kronecker product of \(2 \times 2\) single-qubit matrices, the tensor-product noise model of that literature and the n-ary Kronecker factorization of Section 9.3; each single-qubit matrix is asymmetric in practice (relaxation makes \(|1\rangle \to |0\rangle \) misreads likelier than the reverse), the Z-channel-skewed asymmetric channel above [Smi+21]. The scope of this classical description is wider than classical inputs. If the detector POVM \(\{M_b\}\) is diagonal in the computational basis, \(M_b = \sum _t A(b \mid t)\, |t\rangle \langle t|\), then for any state \(\rho \) the outcome law is \[ \operatorname {Tr}(M_b \rho ) \;=\; \sum _t A(b \mid t)\, \rho _{tt}, \] so the channel applied to the diagonal of \(\rho \) gives the exact outcome statistics whatever the off-diagonal coherences are. What the channel does not describe is the coherence itself, which matters for measurements in other bases and for operations applied before readout, not for the computational-basis outcome law. Section 16.5 marks where a classical channel description stops sufficing.

Axiom transfer and parameter count

Both chapter-4 axioms lift unchanged, being independence claims indifferent to alphabet size: Axiom 1 keeps the events \(\{\text {output on } x_1 = j_1\}\) and \(\{\text {output on } x_2 = j_2\}\) independent for \(x_1 \neq x_2\), and Axiom 2 keeps the \(K\) row distributions of \(Q\) conditionally independent given the objective function. Together they make \(Q\) a sufficient summary, as \((\fprate , \fnrate )\) was in the Boolean case.

A \(K \times L\) row-stochastic matrix has \(KL\) entries and \(K\) row-sum constraints, leaving \[ K(L - 1) \] free parameters, each row a distribution on \(L\) outputs with \(L - 1\) degrees of freedom. This is the n-ary generalization of Corollary 4.4.1.2, recovering the \((\fprate , \fnrate )\) pair at \(K = L = 2\). The count is an upper bound: symmetric structure cuts it further (the BSC’s shared rate drops it from \(2\) to \(1\), the symmetric 3-class matrix above from \(6\) to \(1\)). The contrast with an unconstrained joint over \(K \times L\) outcomes, which carries \(KL - 1\) parameters, is the same parsimony Axioms 1 and 2 bought in chapter 4. Section 9.3 lifts the Kronecker theorem and Section 9.4 the capacity formula, both with the binary \(2\) replaced by \(L\).

9.3 The n-ary Kronecker Theorem

At model order two, chapter 4 proved that the joint confusion matrix of \(m\) queries on a Bernoulli predicate factors as the Kronecker product of the \(m\) per-query matrices (Theorem 4.4.1), with the unconditional form holding in the deterministic-rates regime (Corollary 4.4.1.1) that every worked example in the book inhabits. The same factorization holds for n-ary channels, with the binary output alphabet replaced by one of size \(L\). We drop the conditional star and write \(Q\) for the deterministic per-query matrix.

Queries on different elements that fall in the same block stay independent under Axiom 1; only the row of \(Q\) they consult is shared.

At \(K = L = 2\) this recovers Corollary 4.4.1.2: \(K(L-1) = 2\) and \(K(L^m-1) = 2(2^m-1)\). The reduction sharpens at larger \(L\): a 10-output channel queried 50 times is fixed by \(K \cdot 9\) numbers rather than \(K(10^{50}-1)\).

Worked example

Take the 3-class matrix \(Q\) above and three queries on blocks \((B_1, B_2, B_1)\). The all-correct output \((1, 2, 1)\) has joint probability \[ Q_{1,1} Q_{2,2} Q_{1,1} = 0.9 \cdot 0.9 \cdot 0.9 = 0.729, \] the n-ary analogue of the BSC’s \((1 - \fprate )^m\) joint success. Misclassifying the first query, output \((2, 2, 1)\), gives \[ Q_{1,2} Q_{2,2} Q_{1,1} = 0.05 \cdot 0.9 \cdot 0.9 = 0.0405. \] The joint matrix \(Q^{(3)}\) has shape \(27 \times 27\); the unconstrained joint parameter count \(K(L^m - 1) = 3(27 - 1) = 78\) collapses to the \(K(L-1) = 6\) that specifies \(Q\), and at \(m = 10\) the gap is \(3(3^{10}-1) \approx 1.77 \times 10^5\) against the same \(6\). Section 9.4 turns from structure to the per-use information bound.

9.4 Capacity for n-ary Channels

The BSC capacity \(C = 1 - H(\fprate )\) of Proposition 8.2.1 is the \(K = L = 2\) case of a definition that lifts to any \(K \times L\) channel. The lift is mostly definitional; what changes is that the optimization no longer has a single closed form except for a symmetric sub-family.

The general case

For an n-ary channel (Definition 9.2.1), an input distribution \(P_X\) on the \(K\) blocks induces \(P_Y(j) = \sum _{i} P_X(B_i) Q_{ij}\), and the capacity (8.5) is \begin {equation} \label {eq:n-ary-capacity-def} C \;=\; \max _{P_X}\, \bigl [ H(Y) - H(Y \mid X) \bigr ], \qquad H(Y \mid X) = \sum _{i=1}^{K} P_X(B_i)\, H(Q_{i\,\cdot }), \end {equation} the same object as the BSC’s, now over the \((K-1)\)-simplex rather than the interval \([0,1]\). The mutual information is concave in \(P_X\), so the maximum exists, but the maximizer satisfies implicit stationarity conditions with no closed form in general. The Blahut–Arimoto algorithm, an alternating fixed-point iteration over \(P_X\) and the posterior \(\Prob {X \mid Y}\), converges geometrically to the capacity-achieving distribution (Cover and Thomas [CT06, Ch. 10]), making the general n-ary capacity computable to any precision.

Symmetric channels

The BSC’s closed form rested on two facts: its rows are permutations of one another (so \(H(Y \mid X)\) is input-independent) and its columns are too (so uniform input gives uniform output). Lifting both gives a sub-family with a closed-form capacity.

Call a channel weakly symmetric if every row of \(Q\) is a permutation of a common multiset \(\mathbf {r} = (r_1, \ldots , r_L)\) and every column has the same sum. The first condition makes \(H(Y \mid X) = H(\mathbf {r})\) input-independent; the second makes the uniform input \(P_X(B_i) = 1/K\) produce the uniform output \(P_Y(j) = 1/L\), attaining \(H(Y) = \log _2 L\). (Rows being permutations of one another does not alone uniformize the output: the column sums must match.) Hence \begin {equation} \label {eq:symmetric-capacity} C \;=\; \log _2 L \;-\; H(\mathbf {r}), \end {equation} the n-ary analogue of \(C = 1 - H(\fprate )\): the ceiling \(\log _2 L\) less the irreducible per-use row uncertainty \(H(\mathbf {r})\).

A strongly symmetric channel has, in addition, columns that are permutations of one another. The clean parametrization spreads a single error probability \(p_e\) uniformly over the \(L - 1\) wrong outputs, \[ Q_{ij} = \begin {cases} 1 - p_e & j = i, \\ p_e/(L - 1) & j \neq i, \end {cases} \] a square \(K = L\) matrix with the BSC as its \(L = 2\), \(p_e = \fprate \) instance.

At \(L = 2\), \(p_e = \fprate \), the row \((1 - \fprate , \fprate )\) has entropy \(H(\fprate )\) and \(C = 1 - H(\fprate )\), recovering (8.6). The endpoints carry over: \(p_e = 0\) gives \(C = \log _2 L\) (noiseless \(L\)-ary channel), and \(p_e = (L-1)/L\) gives the uniform row and \(C = 0\).

Worked example

Take \(L = 4\), \(p_e = 0.1\), so each off-diagonal entry is \(0.1/3 \approx 0.0333\) and the channel is strongly symmetric. The ceiling is \(\log _2 4 = 2\) bits. The row entropy is \[ H(\mathbf {r}) = -0.9 \log _2 0.9 - 3 \cdot \tfrac {0.1}{3} \log _2 \tfrac {0.1}{3} \approx 0.137 + 0.490 = 0.627 \text { bits,} \] so \(C = 2 - 0.627 \approx 1.373\) bits per use, well below the noiseless ceiling of \(2\) and well above the BSC’s \(0.531\) at the same per-symbol error \(0.1\). As \(L\) grows at fixed \(p_e\) the error mass spreads more thinly over the wrong outputs. In absolute terms the gap \(H(\mathbf {r}) = H(p_e) + p_e \log _2(L-1)\) (with \(H(p_e)\) the binary entropy) grows with \(L\), but only logarithmically, so the relative capacity \(C/\log _2 L\) tends to \(1 - p_e\). Section 9.5 previews how concrete sketches can each be read as an n-ary channel with its own \(Q\) and its own symmetry.

9.5 Bridge to Sketches

A sketch maps a stream or multiset to a compact summary, often sublinear in the input size, from which a query routine reconstructs an answer with error bounded by an analytic expression in the summary size: the user trades a known accuracy for a known space budget. What distinguishes a sketch from lossy compression is that bounded, calculable error. The three canonical sketches, CountMin for frequency, HyperLogLog for cardinality, and MinHash for similarity, can each be read as a \(\Bernoulli {T}\) value in the sense of this chapter.

The reading summarizes each sketch’s per-query error law as a channel in the sense of Definition 9.2.1, one row per true value. CountMin is a counter (\(T = \mathbb {N}\)) whose one-sided overestimation puts all of \(Q\)’s mass at or above the diagonal; HyperLogLog is a counter whose mass concentrates in a multiplicative band around the diagonal (relative rather than one-sided error); MinHash is a similarity estimator (\(T = [0,1]\)) whose \(Q\) is a binomial kernel with output concentrated in a band of width \(O(1/\sqrt {k})\) at signature length \(k\). Two limits of the reading should be stated before it is used. First, it is a vocabulary, not a new analysis: the constructions and their bounds are those of their authors, cited in Chapter 10. Second, the rows are per-query marginals. Answers to several queries against one stored sketch share its hash functions and counters, so Axiom 1 ( Axiom  1) does not hold across them and the Kronecker factorization of Theorem 9.3.1 does not describe their joint law; the merge rules are sketch-specific monoid operations, not channel compositions. Chapter 10 builds the three from scratch, the first non-Boolean instances of \(\Bernoulli {T}\) the book treats in detail.

9.6 Notes and Lineage

BF Lineage. The n-ary channel is Shannon’s discrete channel with its transition matrix [Sha48]. Capacity as maximal mutual information, the closed forms for symmetric and weakly symmetric channels, and the Blahut–Arimoto iteration are standard [CT06, Chs. 7 and 10], and the Kronecker law for queries on distinct inputs is the product (memoryless) channel. Multi-class confusion matrices and the assignment matrices of quantum readout mitigation [Bra+21; Nat+21] are instances. On the data-structure side, the map-valued counterpart of approximate membership is retrieval: a static function that returns the stored value on keys and an arbitrary value elsewhere, as in the Bloomier filter [Cha+04] and the succinct retrieval structures of Dietzfelbinger and Pagh [DP08], who also reduce approximate membership to retrieval.

BF What this chapter adds. The type-general reading of the noisy bit as \(\Bernoulli {T}\), and the separation of matrix row count from model order, follow the approximate-maps manuscript Towell [Tow26i]. The chapter’s contribution is organizational: classifiers, counters, sketches, and readout share one channel vocabulary. It also corrects that manuscript. Serial composition through a stored stage preserves per-query marginals but not Axiom 1 (Remark 9.3.1), so composition closure of approximate maps holds for marginal rates, and for the joint law only when distinct inputs reach distinct intermediate values or the later stage is re-randomized per query.

BF Left out. Retrieval constructions and their space bounds (the membership side returns in chapters 14 and 15), coding theorems beyond capacity, and the estimation of \(Q\) from data. The source manuscript and Chapter 19 call this object a Bernoulli map; the same name denotes the doubling map \(x \mapsto 2x \bmod 1\) of dynamical systems, so the term is best paired with “approximate” when searching the literature.

Chapter 10
Sketches and Estimators

10.1 Sketches as Bernoulli of \(T\)

A sketch maps a stream or multiset over a universe \(\Universe \) to a compact summary, with a query routine that reconstructs an estimate \(\hat \theta \) of some quantity \(\theta \) of the stream from the summary alone. What makes it a sketch, rather than a compressed representation, is that the summary grows sublinearly (often poly-logarithmically) in the stream length, the per-item update is near-constant, and the error \(\hat \theta - \theta \) is bounded by a clean function of the design parameters (Section 9.5). This chapter builds three, distinguished by what they estimate, which is to say by their output type.

CountMin (Section 10.2) estimates per-item frequency \(f_a\), with \(T = \mathbb {N}\) and one-sided error: the construction guarantees \(\hat f_a \ge f_a\) deterministically, the multiset analogue of the Bloom filter’s one-sided membership report (Chapter 5). HyperLogLog (Section 10.3) estimates distinct-item cardinality \(n\), again with \(T = \mathbb {N}\) but with two-sided relative error of order \(1/\sqrt {m}\) in the register count \(m\). MinHash (Section 10.4) estimates Jaccard similarity \(J(A,B)\), with \(T = [0,1]\) and symmetric error of variance \(J(1-J)/k\) in the signature length \(k\).

The chapter-9 vocabulary gives all three the same shape. Each is a function \(s : \Universe ^* \to T\) whose per-query error can be summarized by a row-stochastic channel \(Q\) (Definition 9.2.1) with rows indexed by the true value \(\theta \) and columns by the report \(\hat \theta \): CountMin’s \(Q\) is upper triangular (one-sided), HyperLogLog’s concentrates in a multiplicative band (relative error), MinHash’s is binomial along each row. This is a reading of known constructions, not a new analysis of them, and it has a definite scope. A row is a per-query marginal, and for CountMin it is conditional on the rest of the stream, since the collision mass depends on the other items’ frequencies. Several queries against one stored sketch share its hash functions and counters, so their errors are dependent: Axiom 1 ( Axiom  1) fails across them, and the Kronecker factorization of Theorem 9.3.1 does not give their joint law. Each of the next three sections follows one template: construction, error bound, \(\Bernoulli {T}\) reading, merge. Section 10.5 records what the three merge rules have in common.

10.2 CountMin

A data stream presents items \(x_1, \ldots , x_n\) from a universe \(\Universe \) with repetition. For \(a \in \Universe \), write \(f_a = |\{i : x_i = a\}|\) for its true frequency and \(F = \sum _b f_b = n\) for the stream length. The frequency-estimation query reports an estimate \(\hat f_a\) of \(f_a\) in a small structure of size polylogarithmic in \(F\), with a tunable error bound.

The construction

Fix a width \(w\) and depth \(d\) and pick \(d\) hash functions \(h_1, \ldots , h_d : \Universe \to \{1, \ldots , w\}\) from a pairwise-independent family (\(\Prob {h_i(a) = u,\, h_i(b) = v} = 1/w^2\) for distinct \(a, b\)), which is cheaper than the full independence the Bloom filter assumed (Definition 5.2.1).

Both operations run in \(O(d)\) time, independent of \(F\) and \(|\Universe |\), in \(d \cdot w\) counters. Each cell \(C[i][h_i(a)]\) holds \(f_a\) plus a non-negative collision mass from other items hashing to the same cell, so \(C[i][h_i(a)] \ge f_a\) for every \(i\) and \(\hat f_a = \min _i C[i][h_i(a)] \ge f_a\). The estimate never under-shoots, the count-valued analogue of the Bloom filter’s deterministic no-false-negatives property; the minimum across \(d\) independent rows attenuates the collision noise, since the estimate exceeds the truth only when every row collides with a heavy item.

The error bound

The bound is additive in \(F\), not in \(f_a\): heavy hitters with \(f_a\) comparable to \(F\) are estimated at small relative error, while light items with \(f_a \ll \epsilon F\) are washed out. CountMin is the sketch for finding which items are heavy, not for the counts of light ones. The space is \(d \cdot w = \lceil \ln (1/\delta ) \rceil \lceil e/\epsilon \rceil = O\big ((1/\epsilon ) \ln (1/\delta )\big )\) counters, logarithmic in the failure probability and inverse in the additive error (Cormode and Muthukrishnan [CM05]).

The \(\Bernoulli {T}\) type and merging

Read through Chapter 9, CountMin is a \(\Bernoulli {T}\) value with \(T = \mathbb {N}\): its channel \(Q\) (Definition 9.2.1) has rows indexed by the true count, and the deterministic \(\hat f_a \ge f_a\) makes \(Q\) upper triangular, the count-valued analogue of the Bloom filter’s one-sided membership matrix. Each row concentrates near the diagonal, with at most a \(\delta \) fraction of mass more than \(\epsilon F\) columns to the right. The row is a marginal for one queried item, conditional on the rest of the stream: the collision mass \(X_i\) depends on the other frequencies, and two queried items that share a cell share its collision mass, so their overestimates are positively dependent. Two sketches built on disjoint substreams with the same parameters and hashes merge by entry-wise addition, \((C_1 \oplus C_2)[i][j] = C_1[i][j] + C_2[i][j]\), which is exactly the sketch of the concatenated stream because Update only increments and addition commutes; the same \((\epsilon , \delta )\) bound holds at the combined \(F\). Sharing the hash family is what makes the merge a single correct sketch rather than two sketches whose failure probabilities add. This is the linearity of the sketch, a property of the construction rather than a channel composition.

Worked example

Take \(w = 10\), \(d = 5\), so the achieved parameters are \(\epsilon = e/w \approx 0.272\) (additive bound \(\epsilon F = 0.272\,F\)) and \(\delta = e^{-d} \approx 0.0067\). On a stream of \(F = 1000\) items with a heavy item \(f_{a_1} = 410\), the expected per-row overshoot is \((F - f_{a_1})/w = 590/10 = 59\), so each row reports near \(469\) and the minimum across five rows falls below it; a typical realization returns \(\hat f_{a_1} = 432\), an overshoot of \(22\), well inside the bound \(\epsilon F = 272\). For a light item with \(f = 4\), the expected per-row overshoot is \(99.6\), and the estimate, though correctly bounded by \(4 + 272\), is uninformative about the true count. This is the heavy-hitter design point: heavy items tight relative to their own count, light items only up to the additive slack. Section 10.3 turns to the complementary question of how many distinct items a stream contains.

10.3 HyperLogLog

The complementary streaming question is cardinality: estimate the number \(n\) of distinct items among \(x_1, \ldots , x_N\), ignoring multiplicity. Exact counting needs \(\Theta (n)\) bits in the worst case; HyperLogLog returns an estimate with relative error of order \(1/\sqrt {m}\) using \(m\) small registers, at space doubly logarithmic in \(n\), so a few kilobytes estimate cardinalities into the billions.

The leading-zero trick

Let \(h : \Universe \to [0,1]\) be a uniform hash, read as a binary fraction \(0.b_1 b_2 \cdots \), and let \(\rho (x)\) be the position of its first \(1\) bit (leading zeros plus one). For uniform \(h(a)\) the bits are fair coins, so \(\Prob {\rho \ge k} = 2^{-(k-1)}\), and the maximum of \(\rho \) over \(n\) distinct items grows as \(\log _2 n\) with \(O(1)\) spread. Tracking only the running maximum of \(\rho (h(a))\) and reporting \(2^{\max }\) therefore estimates \(n\) from a single small integer (the probabilistic counting of Flajolet and Martin [FM85]), but the multiplicative standard deviation is a factor of roughly two: an order-of-magnitude check, useless quantitatively. The fix is averaging.

The construction

HyperLogLog averages without a second hash by stochastic averaging: the first \(\log _2 m\) bits of each item’s hash route it to one of \(m\) buckets, and the remaining bits feed that bucket’s leading-zero count. The \(m\) registers act as independent observations of the single-register estimator on streams of expected length \(n/m\).

The sum \(Z = \sum _j 2^{-M[j]}\) is the harmonic mean of the per-bucket estimates \(2^{M[j]}\) (up to a factor \(m\)), which suppresses an outlier register that drew an unlucky run of leading zeros; the \(m^2\) rescales \(n/m\) per bucket to \(n\) total, and \(\alpha _m\) removes the constant bias of the harmonic mean and the discrete \(\rho \), an integral Flajolet et al. [Fla+07] evaluate. For a 32-bit hash with \(p = 8\) (\(m = 256\)), each \(M[j] \in \{0, \ldots , 24\}\) fits in \(5\) bits, so the sketch is \(256 \cdot 5 = 1280\) bits (\(160\) bytes) regardless of stream length.

The \(1.04\) is the asymptotic per-register factor of Flajolet et al. [Fla+07], from an exact generating-function calculation we do not reproduce. The \(1/\sqrt {m}\) dependence is the central-limit scaling of \(m\) near-independent observations, and doubling the accuracy quadruples the register count. When \(n\) is small relative to \(m\) (many registers still zero) the algorithm switches to a linear-counting correction \(\hat n = m \ln (m/V)\) with \(V\) the count of zero registers; a saturation correction handles the high-cardinality end.

The \(\Bernoulli {T}\) type and merging

Read through Chapter 9, HyperLogLog is a \(\Bernoulli {T}\) value with \(T = \mathbb {N}\), as CountMin is, but its channel \(Q\) (Definition 9.2.1) is not triangular: it errs both ways. The limiting distribution of \(\log \hat n - \log n\) is approximately normal with standard deviation \(1.04/\sqrt {m}\) [Fla+07], so each row of \(Q\) concentrates on a geometric interval around \(n\), symmetric on a multiplicative rather than additive scale, the qualitative contrast with CountMin’s one-sided additive error. Two sketches on the same hash and register count merge by entry-wise maximum, \((M^{(1)} \sqcup M^{(2)})[j] = \max (M^{(1)}[j], M^{(2)}[j])\), which is exactly the sketch of the union stream because each bucket stores a maximum and the max over a union is the max of the per-stream maxima. As with CountMin, sharing the hash is what makes the merge exact; it is a property of the max-register construction, not a channel composition.

Worked example

Take \(m = 256\) and a stream of \(n = 10000\) distinct items. The relative standard error from Proposition 10.3.1 is \(1.04/\sqrt {256} = 0.065\), so a single sketch typically lands within about \(\pm 650\) of \(10000\), with each register holding the maximum \(\rho \) of the \(\approx 10000/256 \approx 39\) items in its bucket. The estimate is nearly unbiased in this regime but high-variance on one seed: averaging \(100\) independent seeds returns an empirical mean of \(9970\) with relative standard deviation \(0.063\), both matching the proposition. The headline figure is a standard error, not a deterministic bound; the value of the sketch is that the spread shrinks as \(1/\sqrt {m}\). Section 10.4 turns to similarity: estimating the Jaccard overlap of two sets from compact signatures.

10.4 MinHash and LSH

The third primitive estimates the Jaccard similarity \(J(A, B) = |A \cap B|/|A \cup B| \in [0,1]\) of two sets. Exact computation needs space linear in the set sizes, a deal-breaker across a corpus of millions of documents over a vocabulary of millions of shingles. MinHash (Broder [Bro97]) reduces each set to a signature of \(k\) integers and estimates \(J\) from two signatures in \(O(k)\) time.

The collision identity

Fix a hash \(h : \Universe \to [0,1]\) that is injective on \(A \cup B\) and min-wise independent there (every element of any subset is equally likely to attain its minimum, as for a uniformly random permutation; Broder et al. [Bro+00]), and write \(\min h(S) = \min _{a \in S} h(a)\).

The minimum hash over a union is a uniform sample from the union, and the chance it falls in a subset is the subset’s proportion. One hash converts the similarity question into a Bernoulli coin of bias \(J\); estimating \(J\) is repetition.

The construction and its error

The signature costs \(O(k)\) space and updates in \(O(1)\) per inserted item via \(\sigma _i \gets \min (\sigma _i, h_i(a))\) (deletion is unsupported, the price of a min-summary).

The error is two-sided and symmetric, like HyperLogLog’s (Proposition 10.3.1) and unlike CountMin’s. Inverting the tail for a target \(\delta \) gives accuracy \(\epsilon = \sqrt {\ln (2/\delta )/(2k)}\), so the signature length grows linearly in \(\ln (1/\delta )\) and quadratically in \(1/\epsilon \).

The \(\Bernoulli {T}\) type

Read through Chapter 9, MinHash is a \(\Bernoulli {T}\) value with \(T = [0,1]\). Since \(k\hat J\) is a sum of \(k\) independent \(\mathrm {Bernoulli}(J)\) variables, each row of the channel \(Q\) (Definition 9.2.1) is a \(\mathrm {Binomial}(k, J)\) distribution on \(\{0, 1/k, \ldots , 1\}\), \[ Q_{J,\, j/k} = \binom {k}{j} J^j (1 - J)^{k-j}, \] symmetric under \(\hat J \mapsto 1 - \hat J\), \(J \mapsto 1 - J\), the contrast with CountMin’s triangular and HyperLogLog’s log-normal rows. The Proposition 10.4.2 concentration is the channel-matrix view of the Binomial’s concentration around its mean. The independence here is across the \(k\) hash functions for one pair of sets; estimates for different pairs that share a set share its signature and are dependent.

LSH for approximate nearest neighbour

Estimating \(J\) for every pair in a corpus of \(N\) documents costs \(O(kN^2)\). Locality-sensitive hashing (Indyk and Motwani [IM98]) uses signatures as bucket keys so that similar pairs collide. Partition the length-\(k\) signature into \(b\) bands of \(r\) rows (\(k = br\)); two sets are candidates if they agree on all \(r\) rows of at least one band. A band agrees with probability \(J^r\), so the candidate probability is the S-curve \[ p(J) = 1 - (1 - J^r)^b, \] with a sharp transition near \(J^* \approx (1/b)^{1/r}\): larger \(r\) steepens and selects, larger \(b\) raises and admits. Tuning \((b, r)\) to a target \(J^*\) and building the \(b\) band tables once turns the \(O(N^2)\) comparison into a sub-linear bucket lookup, with an exact pass filtering the small candidate set.

Worked example

Take \(|A| = |B| = 100\), \(|A \cap B| = 30\), so \(|A \cup B| = 170\) and \(J = 30/170 \approx 0.176\). With \(k = 200\), \[ \mathrm {Var}[\hat J] = \frac {0.176 \cdot 0.824}{200} \approx 7.26 \times 10^{-4}, \] a standard error of \(0.027\). The Hoeffding bound at \(\epsilon = 0.05\) is \(2e^{-2 \cdot 200 \cdot 0.0025}\), that is \(2e^{-1} \approx 0.74\), loose here because Hoeffding ignores the \(J(1-J)\) factor that shrinks the variance for small \(J\); the exact binomial standard error \(0.027\) is the relevant figure. A single sketch returns, say, \(\hat J = 0.185\), within one standard error of \(0.176\). Halving the error to \(0.013\) quadruples the signature to \(k = 800\), the same \(1/\sqrt {k}\) scaling as HyperLogLog. Section 10.5 recaps the common pattern and hands the three sketches to the type algebra of Part III’s close.

10.5 Bridge

Three sketches, one skeleton, differing in output type and channel shape. CountMin (Section 10.2) is the frequency sketch: \(T = \mathbb {N}\), one-sided, an upper-triangular \(Q\) with tail \(\Prob {\hat f_a \ge f_a + \epsilon F} \le \delta \) (Theorem 10.2.1). HyperLogLog (Section 10.3) is the cardinality sketch: \(T = \mathbb {N}\), two-sided and log-normal, with relative error \(1.04/\sqrt {m}\) (Proposition 10.3.1) setting the band width. MinHash (Section 10.4) is the similarity sketch: \(T = [0,1]\), binomial rows, Hoeffding-controlled at signature length \(k\) (Proposition 10.4.2). Each comes with a construction, a row-by-row error bound on \(Q\), and a merge rule.

The merge rules share a structure this chapter notes but does not axiomatize. Merging CountMin tables is entry-wise addition, HyperLogLog registers entry-wise maximum, MinHash signatures (forming the union signature) entry-wise minimum; each is associative, commutative, and has an identity (the zero table, the zero register bank, the all-\(\infty \) signature), and each merge of sketches built with shared hashes equals the sketch of the combined input. That is the mergeability property studied systematically by Agarwal et al. [Aga+13]. It is a property of each construction, a commutative monoid on summaries, and it does not follow from the channel reading: the channel describes one query’s error, while a merge acts on the stored summary. Chapter 13 places the sketches in the same type vocabulary as the rest of Part III without deriving their merge rules from it. Before that, Chapter 11 turns from streaming summaries to randomized algorithms, where a one-sided Monte Carlo procedure is a \(\Bernoulli {\{\text {accept}, \text {reject}\}}\) value and amplification by repetition is independent runs on one input followed by an aggregator (AND, OR, or majority), and Chapter 12 lifts the framework from functions to relations for join and multi-attribute queries.

10.6 Notes and Lineage

BF Lineage. Every construction and bound in the chapter is its authors’. Probabilistic counting from hashed bit patterns is Flajolet and Martin [FM85], and the streaming model with its space lower bounds is Alon, Matias, and Szegedy [AMS99]. CountMin, its \(d \times w\) table of pairwise-independent rows, and the bound of Theorem 10.2.1 at \(w = \lceil e/\epsilon \rceil \), \(d = \lceil \ln (1/\delta ) \rceil \) are Cormode and Muthukrishnan [CM05]. HyperLogLog, the constant \(\alpha _m\), the \(1.04/\sqrt {m}\) error of Proposition 10.3.1, and the limiting log-normal shape are Flajolet et al. [Fla+07]. The MinHash identity \(\Prob {\min h(A) = \min h(B)} = J(A,B)\) and the \(k\)-hash estimator are Broder [Bro97], with the hashing requirement made precise by min-wise independence [Bro+00]; locality-sensitive hashing is Indyk and Motwani [IM98]. That merges of sketches with shared hashes equal sketches of the combined input is the mergeable-summaries property of Agarwal et al. [Aga+13].

BF What this chapter adds. A common reading of the three sketches as \(\Bernoulli {T}\) values whose per-query error law is a row of a channel matrix (triangular, multiplicative-band, binomial), and an explicit statement of where that reading stops: rows are marginals, CountMin’s are conditional on the rest of the stream, and queries against one stored sketch are dependent, so the Kronecker factorization of chapter 9 does not describe them.

BF Left out. Heavy-hitter, quantile, and inner-product applications; Count-Sketch and other linear sketches; bias corrections and sparse representations of HyperLogLog; lower bounds for the three problems; and the generating-function analysis behind \(\alpha _m\) and \(1.04\), which is quoted rather than reproduced.

Chapter 11
Randomized Algorithms

11.1 Miller-Rabin Formal

Chapter 3 §3.4 (Section 3.5) walked through the Miller-Rabin primality test as a \(\Bernoulli {\BoolType }\) instance, stating the per-round bound \(\fprate \le 1/4\) as a fact and applying it. This section closes that deferral, restating the walkthrough as one formal theorem with the per-round bound credited to Rabin and the \(k\)-round bound derived from the composition theorem.

Fix an odd \(n > 3\), write \(n - 1 = 2^r d\) with \(d\) odd, and let \(W_n(a)\) be the single-round witness test on \(a \in \{2, \ldots , n-2\}\), returning \(\top \) (probably-prime) or \(\bot \) (composite), with the predicate “\(n\) is prime.”

The division of labor is the one chapter 3 promised: number theory supplies the strong-liar bound, the framework supplies the rest. The two error directions take the two faces of the composition theorem, \(1 - \prod (1 - \omega _i)\) for the monotone false-negative direction and \(\prod \varepsilon _i\) for the all-error false-positive direction, exactly as §3.3 distinguishes them. Choosing \(k\) to drive \(\fprate ^{(k)}\) below a target \(\delta \) is the sample-complexity question of Section 11.4.

11.2 Las Vegas vs Monte Carlo

Randomized algorithms divide by where the randomness surfaces: in the runtime, with the output exactly correct, or in the output, with the runtime predictably bounded.

The two are exhaustive of the useful regimes: an algorithm that is both deterministically correct and worst-case bounded is just deterministic, and one that is neither offers nothing. They are the two faces of one trade-off, since a worst-case runtime bound over all coin sequences leaves no room to keep flipping until the answer is right: if every sequence halts by step \(T(|x|)\) and the answer in hand at that step is wrong, the algorithm must return it. Each regime gives up one guarantee. Las Vegas keeps the output exact and pays in the runtime tail; Monte Carlo keeps the runtime fixed and pays in the output. Which to pay on is an application question, exact-but-eventually versus fast-but-occasionally-wrong.

BF Three examples. Randomized quicksort (Las Vegas) sorts by partitioning around a uniform random pivot: the output is always correctly sorted, while the runtime is \(O(n\log n)\) in expectation with a \(\Theta (n^2)\) tail of probability \(n^{-\Omega (c)}\) at \(cn\log n\) comparisons. Karger min-cut (Monte Carlo) contracts uniform random edges until two super-vertices remain, in deterministic \(O(n^2)\) time, returning a fixed min-cut with probability only \(\ge 2/[n(n-1)]\); running it \(r\) times and keeping the smallest cut fails with probability \(\le (1 - 2/[n(n-1)])^r \le \exp (-2r/[n(n-1)])\), so \(r \ge \tfrac {n(n-1)}{2}\ln (1/\delta )\) trials suffice, a cubic-in-\(n\) cost that is the price of a per-trial signal as weak as \(1/n^2\). Miller-Rabin (Monte Carlo, Section 11.1) runs in \(O(k\log ^3 n)\) and errs only on composites, at rate \(\le (1/4)^k\) by Theorem 11.1.1; its per-round signal \(1/4\) is independent of input size, so amplification needs only \(O(\log (1/\delta ))\) rounds, the opposite extreme from Karger.

BF Conversion. The regimes interconvert at a cost. A Las Vegas algorithm with expected runtime \(\mu (|x|)\) becomes Monte Carlo by running for \(c\,\mu (|x|)\) steps and reporting failure on timeout: by Markov the failure probability is \(\le 1/c\), so \(c = 10\) gives error \(\le 0.1\) at worst-case runtime \(10\mu (|x|)\). This is why expected polynomial time is essentially worst-case polynomial time with bounded error. The reverse needs a verifier: a Monte Carlo algorithm with a deterministic polynomial-time check of candidate outputs becomes Las Vegas by repeating until a verified result appears, in expected \(1/(1-\fprate )\) trials. The verifier is a nontrivial assumption, which is why \(\mathrm {ZPP} = \mathrm {RP} \cap \mathrm {coRP}\) (Section 11.3) but the directions are not symmetric.

BF The \(\Bernoulli {\BoolType }\) correspondence. A Monte Carlo decision algorithm returns a Boolean \(A(x)\) guessing a predicate \(P(x)\), with error structure exactly the \(2 \times 2\) channel of Definition 4.5.1, \(\fprate = \Prob {A(x) = \top \mid P(x) = \bot }\) and \(\fnrate = \Prob {A(x) = \bot \mid P(x) = \top }\), with one qualification: the error probability of a randomized algorithm depends on the input \(x\), not only on the truth value \(P(x)\), so the rates are per input and the class definitions of Section 11.3 bound them uniformly. Read with that qualification, as a worst-case pair in the sense of Proposition 3.3.2 (see also Section 16.2), it is a \(\Bernoulli {\BoolType }\) instance; the \(k\)-fold amplification of Theorem 11.1.1 is the composition theorem (Theorem 3.4.1) on independent runs over one fixed input; and chapter 7’s PPV and NPV apply with prevalence the base rate of true positives. A Las Vegas algorithm has no error to rate, its output being deterministically correct and its randomness living in the runtime; its analysis belongs to runtime tail bounds (Markov, Chebyshev, Chernoff of Section 11.4), not to the Bernoulli channel. From here, “randomized algorithm” means a Monte Carlo decision algorithm, read as a \(\Bernoulli {\BoolType }\) instance.

11.3 RP, BPP, and Bernoulli Type

The Monte Carlo decision problems (Definition 11.2.2) carve into the classes RP, coRP, ZPP, and BPP by where the error is allowed to land, the one-sided and two-sided classification of Gill [Gil77]. Each is a location on the chapter-6 rate plane: RP and coRP on the two axes of the perfect filter line, ZPP at the origin, BPP in the interior. The rate-plane picture restates Gill’s classification in the coordinates of this book; the class definitions bound the per-input rates uniformly over inputs.

RP: certain accept, the FPR-zero axis

Read \(A(x) = 1\) as the report in-\(L\). The second line says the algorithm never accepts a non-member, \(\fprate = \Prob {A(x) = 1 \mid x \notin L} = 0\); the first says it accepts a member with probability \(\ge 1/2\), \(\fnrate = \Prob {A(x) = 0 \mid x \in L} \le 1/2\). An RP algorithm is a \(\Bernoulli {\BoolType }\) instance with \(\fprate = 0\), \(\fnrate \le 1/2\): it sits on the FPR-zero axis of the perfect filter line (Definition 6.4.1). An accept is certain, a witness seen is a witness believed, and the only failure is missing a witness that exists. This is the certain-accept shape, the decision-algorithm kin of the perfect hash filter of Chapter 15, not the Bloom filter, whose certainty is on the reject side.

The canonical RP problem is Non-Zero Polynomial: given a circuit for a polynomial \(f\) over a finite field, decide whether \(f \not \equiv 0\). By Schwartz–Zippel, a random assignment makes a non-zero \(f\) evaluate non-zero with probability \(\ge 1 - d/|F|\) (degree \(d\), field size \(|F|\)), while a zero \(f\) always evaluates zero. Reporting non-zero on a non-zero evaluation is the RP shape: silent on zero polynomials (\(\fprate = 0\)), occasionally missing a non-zero one (\(\fnrate \le d/|F|\)).

coRP and ZPP: the mirror axis and the origin

A coRP algorithm has \(\fnrate = 0\), \(\fprate \le 1/2\): it sits on the FNR-zero axis, the certain-reject shape, the decision-algorithm kin of the Bloom filter (Definition 5.2.1), whose \(\bot \) report is certain. Miller-Rabin (Theorem 11.1.1) is the canonical example: a composite witness certifies compositeness, so on the language Primes a reject is certain (\(\fnrate = 0\), a prime is never rejected) while a composite is occasionally accepted as prime (\(\fprate \le 1/4\)). Thus \(\textsc {Primes} \in \mathrm {coRP}\), and dually \(\textsc {Composites} \in \mathrm {RP}\) via the same algorithm, where instead a composite-witness is a certain accept of compositeness. (This was the textbook RP/coRP example until the deterministic AKS test placed Primes in P [AKS04].) ZPP, at \(\fprate = \fnrate = 0\), is the origin: exact decisions paid for in the runtime tail, equivalently the expected-polynomial-time Las Vegas class.

BPP: the interior

Read as a channel, \(\fnrate , \fprate \le 1/3\): BPP sits in the interior of the rate plane, off both axes, the two-sided case. The chapter-7 vocabulary applies directly. At prevalence \(\pi = \Prob {x \in L}\) the positive predictive value (Definition 7.4.1) is \[ \PPV = \frac {\pi (1 - \fnrate )}{\pi (1 - \fnrate ) + (1 - \pi )\fprate } \ge \frac {\pi \cdot 2/3}{\pi \cdot 2/3 + (1 - \pi )\cdot 1/3} = \frac {2\pi }{\pi + 1}, \] which at \(\pi = 0.01\) is only \(\approx 0.0198\): despite \(2/3\)-versus-\(1/3\) per-class accuracy, \(98\%\) of accept reports are false at low prevalence, the same prevalence-dependence Definition 7.4.2 and chapter 7 record. The \(2/3\)-versus- \(1/3\) gap is conventional but load-bearing: it forces a signal in both directions so amplification can drive both errors down, where the gap-closed class PP cannot.

Amplification

This is the composition theorem (Theorem 3.4.1) with the OR aggregator: the certain-accept property (\(\fprate = 0\)) is preserved (any accept is true), and, since every run sees the same member \(x \in L\), \(\fnrate \) at the OR is the product of the per-run rates by independence of the runs’ coins (independence across stages, Remark 3.4.1, not Axiom 1). It is the FPR-zero mirror of the \(k\)-round Miller-Rabin bound, which drives the coRP rate \(\fprate \) down via AND; coRP amplifies dually. Bringing \(\fnrate \) to \(2^{-30}\) costs \(30\) runs.

The exponent is no longer the per-run rate but the Chernoff exponent for a majority of independent draws; the proof, via the Chernoff bound (Theorem 11.4.1), is in Section 11.4. BPP error decays exponentially in the repetition count even though no single coin sequence drives either error to zero.

The correspondence at scale

The first three rows trace the perfect filter line of Definition 6.4.1: RP the FPR-zero axis, coRP the FNR-zero axis, ZPP their intersection at the origin. BPP is the interior below \(1/3\). The chapter-6 classification of approximate sets by axis and the randomized complexity taxonomy apply one distinction to two families, arbitrary \(\Bernoulli {\BoolType }\) instances there and polynomially constrained decision algorithms here; the one-sided-versus-two-sided distinction of Section 5.4 is what separates RP and coRP from BPP. Amplification is what the two families share when, and only when, independent re-randomization is available: any instance with per-trial error bounded away from \(1/2\) on both inputs is driven to negligible error by independent repetition on the same input and the right aggregator (OR or AND on an axis, majority in the interior). A stored approximate set cannot be amplified this way, since re-querying it returns the same answer; it must be rebuilt with fresh randomness.

11.4 Sample Complexity

Every algorithm of this chapter has a knob \(k\), the round or repetition count, that drives a per-trial error below a target \(\delta \). The sample-complexity question is how to set it, and the answer rests on one theorem about sums of independent Bernoulli variables.

The upper tail’s \(\mu \epsilon ^2/3\) is looser than the lower tail’s \(\mu \epsilon ^2/2\) (deviations above the mean are harder against the \(\{0,1\}\) ceiling); a two-sided union gives \(\Prob {|X - \mu | \ge \epsilon \mu } \le 2\exp (-\mu \epsilon ^2/3)\).

BF Miller-Rabin round count. Miller-Rabin sits on the FNR-zero axis (\(\fnrate = 0\), \(\fprate \le 1/4\)), so the Chernoff bound is not needed: a single fooled round suffices for a false positive, and the composition theorem reduces \(\fprate ^{(k)}\) to the product \(\le (1/4)^k\) (Theorem 11.1.1). Driving this below \(\delta \) requires \(k \ge \log _4(1/\delta ) = \tfrac 12\log _2(1/\delta )\): about \(10\) rounds for \(\delta = 10^{-6}\), \(20\) for \(10^{-12}\), \(64\) for \(2^{-128}\). Each bit of confidence costs a constant number of rounds, not a constant factor, because the error is one-sided and a single witness collapses the chain.

BF BPP amplification. The deferred proof of Theorem 11.3.2 is a Chernoff calculation. Let \(A\) accept correctly with probability \(\ge 1/2 + \epsilon _0\) (\(\epsilon _0 = 1/6\) for the \(2/3\)-versus-\(1/3\) gap of Definition 11.3.3), let \(X_i = 1\) when run \(i\) is correct, independent across runs because the runs use independent coins on the same input, and let \(X = \sum _{i=1}^k X_i\) with \(\mu = k(1/2 + \epsilon _0)\). The majority is wrong when \(X < k/2 = (1 - \epsilon ')\mu \) with \(\epsilon ' = \epsilon _0/(1/2 + \epsilon _0)\), so the lower tail gives \[ \Prob {X < k/2} \le \exp \!\left (-\tfrac {\mu (\epsilon ')^2}{2}\right ) = \exp \!\left (-\tfrac {k\epsilon _0^2}{2(1/2 + \epsilon _0)}\right ) \le \exp \!\left (-\tfrac {k\epsilon _0^2}{3}\right ), \] the last step since \(1/[2(1/2 + \epsilon _0)] \ge 1/3\) for \(\epsilon _0 \le 1\). Asking this be \(\le \delta \) gives \(k \ge 3\ln (1/\delta )/\epsilon _0^2\), so the \(\epsilon _0 = 1/6\) gap costs \(108\ln (1/\delta )\) rounds (the constant tightens with the sharper lower-tail exponent). The point is the shape: BPP amplification is \(O((1/\epsilon _0^2)\log (1/\delta ))\), exponential decay in \(k\) with the exponent the squared margin above \(1/2\). Against RP amplification’s bit-per-round (\(\fnrate \) from \(1/2\) to \(2^{-k}\) in \(k\) rounds), the two-sided contract pays a \(\Theta (1/\epsilon _0^2)\) overhead per bit, the price of needing a vote rather than a single witness.

BF Chernoff versus the variance interval. The Chernoff bound and the variance interval of Section 7.2 are two tails of the same binomial estimator \(\hat p = X/n\). Chebyshev on the variance \(p(1-p)/n\) (Proposition 7.2.2) gives \(\Prob {|\hat p - p| \ge \epsilon } \le p(1-p)/(n\epsilon ^2) \le 1/(4n\epsilon ^2)\), polynomial in \(n\epsilon ^2\); Hoeffding, the additive cousin of Theorem 11.4.1, gives \(\le 2\exp (-2n\epsilon ^2)\) [Hoe63], exponential. Chebyshev needs only the variance, the quantity chapter 7’s estimator analysis computes; Chernoff and Hoeffding need the full moment-generating function and give sample complexity. At a common accuracy \(\epsilon \) the requirement is \(n \ge 1/(4\epsilon ^2\delta )\) from Chebyshev against \(n \ge \ln (2/\delta )/(2\epsilon ^2)\) from Hoeffding. At \(\delta = 10^{-6}\) that is about \(2.5 \times 10^5/\epsilon ^2\) samples against about \(7.3/\epsilon ^2\). This is why exponential tail bounds, not Chebyshev, are the workhorse of randomized-algorithm analysis.

11.5 Bridge

The chapter traced one figure from four angles. Section 11.1 closed the chapter-3 deferral, making Miller-Rabin a formal \(\Bernoulli {\BoolType }\) with \(\fnrate = 0\) and \(\fprate \le 1/4\) per round (Theorem 11.1.1). Section 11.2 separated the Monte Carlo regime, where \(\Bernoulli {\BoolType }\) analysis applies, from its exact-output Las Vegas sibling. Section 11.3 read the Monte Carlo classes as locations on the rate plane of Definition 6.4.1: RP (Definition 11.3.1) on the FPR-zero axis (certain accept), coRP on the FNR-zero axis (certain reject, the Bloom-filter shape), ZPP at the origin, BPP in the interior, with the one-sided-versus-two-sided split of Section 5.4 separating RP and coRP from BPP. Section 11.4 quantified the repetition count, the Chernoff bound (Theorem 11.4.1) giving the \(k = \Theta (\log (1/\delta ))\) that makes randomized algorithms practical and sharpening the chapter-7 variance interval (Section 7.2) from polynomial to exponential decay.

The picture extends in two directions before Part III closes. Chapter 12 lifts the analysis to binary relations, where the question is whether \(x\) and \(y\) stand in a relation rather than whether \(x\) belongs to a set, the setting for joins and pair predicates; a Monte Carlo relation test is a \(\Bernoulli {\BoolType }\) value per pair, and amplification by independent runs on one pair carries over, while a stored approximate relation, like a stored approximate set, must be rebuilt rather than re-queried to gain independence. Chapter 13 closes Part III by placing the chapter-10 sketches, this chapter’s randomized algorithms, and the chapter-12 relations in one type vocabulary over \(\Bernoulli {T}\), and by recording which independence hypotheses make their joint error laws factor.

11.6 Notes and Lineage

BF Lineage. The witness test and its conditional deterministic form are Miller [Mil76]; the unconditional strong-liar bound \(\fprate \le 1/4\) behind Theorem 11.1.1 is Rabin [Rab80], and Agrawal, Kayal, and Saxena [AKS04] later placed primality in P. The classes RP, coRP, ZPP, and BPP, their one-sided and two-sided error conditions, and ZPP \(=\) RP \(\cap \) coRP are Gill [Gil77]. The exponential tail bounds are Chernoff [Che52] and Hoeffding [Hoe63]; the forms used here, and the majority-vote amplification of BPP, follow Mitzenmacher and Upfal [MU17, Ch. 4], with Motwani and Raghavan [MR95] the standard treatment of Las Vegas and Monte Carlo algorithms and Cormen et al. [Cor+09, Ch. 31] for the number theory. The contraction algorithm is Karger [Kar93]. Amplitude amplification is Grover [Gro96] for search and Brassard et al. [Bra+02] in general.

BF What this chapter adds. Nothing in the chapter is a new theorem. It restates the classical taxonomy in the rate coordinates of the rest of the book (RP and coRP on the two axes of the perfect filter line, BPP in the interior), with two clarifications that matter for the framework: the rates of a randomized algorithm are per input, so the class definitions supply worst-case pairs rather than constant rates, and the independence that amplification uses is independence across runs on one input, not Axiom 1 across elements. The comparison of OR-amplification with amplitude amplification is an exposition of the \(1/p\) versus \(1/\sqrt {p}\) scaling.

BF Left out. Derandomization, the relations among BPP, the polynomial hierarchy, and BQP, pseudorandom generators, and the number theory of the strong-liar bound, which is cited rather than proved.

Chapter 12
Approximate Relations

12.1 From Approximate Sets to Approximate Relations

A binary relation \(R \subseteq T \times U\) is a set whose elements are ordered pairs, so the chapter-5 construction (Chapter 5) applies with the universe \(T \times U\) in place of \(\Universe \); relations of higher arity live the same way on a product of attribute domains. We use finite relations with set semantics in the sense of Codd [Cod70]: no duplicate rows, no nulls, and no approximate comparison of attribute values. A relation is its membership predicate \(\SetIndicator {R}\) (Section 5.1), and an approximate relation is a noisy version of that predicate, \(\tilde R : T \times U \to \Bernoulli {\BoolType }\), with the two rates of Definition 4.5.1.

What the set-level chapters could leave implicit must now be said explicitly, because relational operators reuse their inputs. Three things are kept apart: the exact relation, which is fixed and carries no prior; the construction randomness, sampled once when the structure is built; and the source identity of each probe, the stored random outcome that the probe reads.

These are pointwise construction probabilities with the relation and the query held fixed. They are not the probability that an observed positive is false (that is the predictive value of Chapter 7), and repeating one probe \(N\) times supplies one outcome, not \(N\) trials.

Every occurrence of a source reads the same bit, and distinct relations may share sources. A symmetry-preserving structure that answers \(R(a, b)\) and \(R(b, a)\) from one canonical key is the motivating case: the query \(R(a, b) \land R(b, a)\) then has the single source \(X_{\{a, b\}}\) as its lineage, not a conjunction of two independent bits. The independent-tuple model is an idealized channel, not a consequence of storing pairs in a Bloom filter: shared bit arrays and finite hash families introduce dependence (Chapter 16). Shared storage is also different from shared outcomes. In the perfect-hash-plus-fingerprint construction (Chapter 15), with an ideal fingerprint function independent of the placement, two absent keys that land in the same slot still match it independently, each with probability \(2^{-M}\); a repeated canonical key, by contrast, reads one outcome twice.

Two examples fix the picture. The friend-of relation \(F \subseteq P \times P\) on a population \(P\) stores known friend pairs in a Bloom-style structure: a probe \(\tilde F(\text {Alice}, \text {Bob})\) reads the hashed positions for the pair, with \(\fnrate = 0\) and a tunable \(\fprate \), the chapter-5 analysis applied to pairs. Conditional on the filter’s occupancy, distinct absent pairs test positive independently at a common rate; unconditionally they are dependent through that occupancy, a distinction Section 12.4 turns into a usable bound. The word-in-document relation \(W \times D\), the inverted index of information retrieval, holds \((w, d)\) when word \(w\) occurs in document \(d\), representable as one Bloom filter per document.

The chapter-9 view (Chapter 9, Section 9.2) is equivalent: \(\SetIndicator {R}\) is a binary function \(T \times U \to \BoolType \) whose channel has one row per input pair, collapsing under Axiom 2 to model order two (in the relation or not), exactly as chapter 5 collapsed the per-element rates of an approximate set. Section 12.2 adds the relational operators; Section 12.3 asks which structural invariants an approximation can keep; Section 12.4 states what error guarantee a whole query inherits from its sources; and Section 12.5 reads the MinHash similarity join of chapter 10 in the same vocabulary.

12.2 Composition of Approximate Relations

Selection, projection, and join build new relations from old ones. Applied to approximate operands, each operator induces a derived approximate relation whose membership test is a Boolean function, its lineage, of the operand tests. Every operator raises two questions: what are the derived marginal rates, and are the derived tuples still independent? The answers depend on which sources the outputs read (Definition 12.1.2). Unless stated otherwise, the two operands of a binary operator are built independently of each other (cross-relation independence), each satisfies the independent-tuple model, and every predicate in an operator (the selection condition, the join condition on attribute values) is exact: the only randomness is inherited from the operand tests.

Selection

Selection \(\sigma _P(R) = \{t \in R : P(t)\}\) with an exact predicate \(P\) is the operand read on the smaller universe \(\{t : P(t)\}\). It keeps the rates within each existing truth class and keeps independence under the independent-tuple model; for a source-sharing input it keeps whatever dependence the selected tuples already had. Selection preserves the rate partition: it does not turn a two-rate model into a one-rate model. Viewed over the original universe, the tuples outside \(\{t : P(t)\}\) are deterministic negatives. When \(P\) is itself approximate, the selection is the AND of two noisy tests at the same tuple and falls under the join analysis below.

Projection

The projection \(\pi _Y\) onto an attribute subset \(Y\) collapses the remaining attributes by an existential quantifier. The fiber \(\phi (s)\) of a projected tuple \(s\) is the set of candidate tuples projecting to it, its size \(k(s)\) is the fan-in, and \(j(s) = \Card {\phi (s) \cap R}\) counts its true members. The target universe is the image of the candidate universe, so every fiber is nonempty. Projection is a logical OR over the fiber.

The false-positive side amplifies with the fan-in, and the true-positive side is boosted, because a nonmember fiber tuple can false-positive and so rescue a genuine projected tuple. On a full Cartesian candidate universe a coordinate projection has constant fan-in, the product of the discarded domain sizes, but the true-member count still varies: at \((\fprate , \tprate ) = (0.05, 0.90)\), fibers with \((k, j) = (3, 1)\) and \((3, 2)\) have true-positive rates \(0.90975\) and \(0.9905\). Uniform fan-in alone does not give a common true-positive rate. In the book’s vocabulary the projection raises the model order, one rate block per attained profile (several profiles may share a rate), while independence is a separate question.

That question has a different answer once sources are shared. Let \(A_s\) be the set of distinct sources read by the fiber of \(s\), so that \(Y_s = \bigvee _{i \in A_s} X_i\).

At boundary rates an overlapping support can still give zero covariance; the formula holds, but the disjointness criterion needs interior rates. The smallest case shows that distinct tuples need not be distinct trials: on the candidate universe \(\{(a, b), (b, a)\}\) with \(a \ne b\), a canonical symmetric test reads one source \(Z\) for both tuples, projection onto the first coordinate has disjoint tuple fibers, and yet both outputs equal \(Z\), with covariance \(p(1 - p)\).

Natural join

The natural join of \(R_1 \subseteq T \times U\) and \(R_2 \subseteq U \times V\) is \(R_1 \bowtie R_2 = \{(t, u, v) : (t, u) \in R_1,\, (u, v) \in R_2\}\), and its approximate version is the AND of two noisy tests, \((\tilde R_1 \bowtie \tilde R_2)(t, u, v) = \tilde R_1(t, u) \land \tilde R_2(u, v)\), the object of Section 3.1.

Without a distribution on queries, or counts of the three regions, the three pointwise regimes remain distinct: a join output is not a two-rate tuple. The bare product \(\fprate _1 \fprate _2\) is only the both-absent corner. It is a good summary when out-of-join triples are dominated by that corner (both relations sparse in a large triple universe), and it understates the error wherever half-matched triples carry weight, since there the surviving single rate applies.

A join breaks element-wise independence

Theorem 12.2.3 describes one output triple. Output triples that share an operand tuple read the same trial.

At \(a \in \{0, 1\}\), or when a partner never fires, the covariance vanishes: the theorem identifies a source-reuse mechanism, not a claim that every join produces dependence. As a concrete reading, take two true triples sharing their \(R_1\)-pair, with every operand true-positive rate \(0.9\). They are accepted together with probability \(0.9^3 = 0.729\), while an independence assumption predicts \(0.9^4 = 0.6561\); the covariance is \(0.0729\) and the joint probability is \(1/0.9 \approx 1.11\) times the product of marginals. A planner that multiplied per-row survival probabilities would understate the chance that both rows survive. The marginal rate of any single output is still exact, and an expected false-positive count over a join still adds tuple by tuple, since linearity of expectation needs no independence; the higher moments and any all-correct probability computed as a product over output tuples are what the correlation affects.

Worked example

Let \(\tilde F\) be a friend-of relation with \(\fnrate _F = 0\), \(\fprate _F = 0.05\), and \(\tilde C\) a coworker relation with \(\fnrate _C = 0\), \(\fprate _C = 0.02\), built and queried independently. The join returns triples \((p_1, p_2, p_3)\) with \(p_1\) reported a friend of \(p_2\) and \(p_2\) reported a coworker of \(p_3\). A triple with both pairs absent is accepted with probability \(\fprate _F \fprate _C = 0.001\), three orders below either operand, the relational face of the intersection gain of chapter 6. A triple in which \(p_1\) truly is a friend of \(p_2\) but \(p_2\) is not a coworker of \(p_3\) is accepted with probability \(\fprate _C = 0.02\), so where such half-matched triples are common the average sits nearer to \(\max (\fprate _F, \fprate _C)\) than to the product. The gain rests on cross-relation independence (a shared hash family breaks the product) and on sparsity.

Composing the operators: lineage, not rate chaining

A query built from selection, projection, join, and union assigns each candidate answer a monotone Boolean lineage over source bits, conjunction combining the facts one witness needs and disjunction combining alternative witnesses. This is established provenance machinery [GKT07; Suc+11]. With the exact truth profile fixed and aliases identified, the answer’s marginal probability under the independent-source model is a polynomial in the source rates read off that lineage, the relational form of the per-latent-assignment propagation of Proposition 3.3.1.

What does not work is chaining a two-rate update per operator up the expression tree. Three failures recur. First, a join output is not a two-rate tuple (Theorem 12.2.3): feeding one summary rate into a downstream operator reproduces the error of Example 3.1. Second, in join-then-project the fiber of a projected tuple consists of join outputs that share operand tuples, so its OR is not over independent trials and \(1 - (1 - \fprate )^k\) does not apply; the source supports of Theorem 12.2.2 must be carried instead. Third, repeated sources are not fresh trials: a conjunction reading one absent source eight times at rate \(0.05\) fires with probability \(0.05\), whereas treating the occurrences as independent gives \(0.05^8 \approx 3.9 \times 10^{-11}\). What survives is exact lineage evaluation for a fixed profile, linearity of expectation for counts, the union bound over a fixed finite workload without any independence, and the truth-uniform contracts of Section 12.4. The probability that a whole result is error-free does not factor over output tuples once they share sources. The projection penalty of a large fan-in is also what motivates the sketch-based similarity join of Section 12.5.

12.3 Preservation of Relational Invariants

A relation may be required to be reflexive, symmetric, single-valued, or transitive in every realization. Per-tuple rates alone do not secure such global properties. We first compute preservation probabilities under the independent-tuple model, then compare three ways to enforce a constraint: sharing a canonical source, adding reported tuples, and deleting them. The three have different rate and dependence consequences; they are specific constructions, not a classification of all repairs. Throughout, \(R \subseteq A \times A\) (or \(X \times Y\)) is fixed and has the named property, and \(\tilde R\) has rates \((\fprate , \tprate )\) with \(\tprate = 1 - \fnrate \).

Reflexivity

When reflexivity is known in advance, answering \(\top \) on the diagonal enforces it exactly, with diagonal true-positive rate one and no false positive there. That is a rule for known forced tuples, not an OR over a singleton trial.

Symmetry and its closure

For \(a \ne b\), symmetry requires the ordered bits of \((a, b)\) and \((b, a)\) to agree; diagonal pairs impose no constraint.

The symmetric closure forces agreement by admitting a pair when either ordered trial fires.

At \(\tprate = 0.9\) the off-diagonal true-positive rate becomes \(0.99\), paid for by a false-positive rate of \(1 - 0.95^2 = 0.0975\) at \(\fprate = 0.05\). The two repaired ordered bits within a pair are identical, hence dependent whenever their common acceptance probability is interior; independence remains only between distinct unordered pairs, and a later operator must use these shared source identities (Theorem 12.2.2).

Single-valuedness

A relation \(f \subseteq X \times Y\) is single-valued when each key has at most one reported value. Having no spurious value is sufficient but not necessary: if the true value is missed, one spurious value still leaves a single-valued row. Keys without a true value must be counted too.

The familiar factor \((1 - \fprate )^{v - 1}\) is the probability of no spurious report at a defined key; it equals that row’s single-valuedness probability only when \(\fnrate = 0\).

Transitivity

Transitivity requires every reported two-step path to have a reported shortcut. The premises are reported pairs, not merely true ones, and the triple constraints overlap, so their probabilities cannot be multiplied in general. A disjoint subfamily still gives a clean bound.

The interior-rate hypothesis matters: an empty exact relation with \(\fprate = 0\) stays empty and transitive surely. There is no universal ordering of the four invariants by preservation probability, and the bound implies nothing about the cost of repairing transitivity.

Canonical sources and specific repairs

Fix an equivalence on the tuple universe and call a constraint class-induced when its satisfying relations are exactly the unions of equivalence classes (orbits). Every orbit of an admissible exact relation is then homogeneous, wholly inside or wholly outside \(R\). Symmetry is class-induced, with orbits \(\{(a, b), (b, a)\}\) off the diagonal and singletons on it. Reflexivity is not (it forces specified tuples), and single-valuedness is a cardinality cap per row; both are handled separately.

A bound \(h(O) \le m\) gives only bounds on the orbit rates; equality holds at the actual orbit size. The bits within an orbit coincide, so tuple independence is lost except on singleton or deterministic orbits.

The invariant therefore costs nothing in marginal rates. Building the structure over the quotient universe also reduces the stored keys, to \(|R|/2\) for a symmetric relation with empty diagonal, although encoding and fixed overheads keep that from being a guaranteed halving of bytes. The price moves elsewhere: the common orbit answer is a shared source in the sense of Definition 12.1.2, so later operators see dependent tuples, and Section 12.4 shows the effect on query error and on the cost of exact inference. The OR-closure has a different purpose, raising recall at a fixed raw rate while raising the false-positive rate too, and its increase can be budgeted.

An accounting model that charges \(-\log _2 \fprate \) bits per stored key, other costs fixed, therefore assigns an asymptotic surcharge of \(\log _2 m\) bits per key. This is conditional accounting, not a memory lower bound for two-sided channels: the one-sided membership bound of Chapter 8 has \(\tprate = 1\) and cannot price a recall improvement.

Deletion enforces single-valuedness instead: keep at most one reported value per key, breaking ties uniformly with randomness independent of the trials, sampled once per row and reused. A defined key then retains its true value with probability \[ \tprate \, \Expect {(1 + K)^{-1}} = \tprate \, \frac {1 - (1 - \fprate )^{v}}{v \fprate }, \qquad K \sim \mathrm {Binomial}(v - 1, \fprate ),\ \fprate > 0, \] with continuous extension \(\tprate \) at \(\fprate = 0\), using \(\Expect {1/(1 + K)} = \int _0^1 (1 - \fprate + \fprate x)^{v - 1}\, dx\). The retained rate decays as \(v\) grows; deletion never raises a false-positive probability but can discard the true value. A map representation enforces single-valuedness by its output type instead (Chapter 9).

Transitivity is outside the orbit construction altogether.

The lemma excludes this particular canonical-source construction; it excludes neither other encodings, nor restricted subclasses, nor repairs that add or delete tuples. One such addition rule is easy to analyze exactly.

The iteration thus has no universe-independent bound on its rounds and can add \(\Theta (k^2)\) pairs, a statement about this simultaneous addition rule only. Other closure algorithms and repair objectives behave differently; deleting every pair, for instance, yields a transitive relation at the cost of every true answer. The dependence of repair problems on the constraint class is studied at length in database repair and consistent query answering [Ber11].

What the constructions change

The constructions expose how a representation choice moves error between three places: the marginal rates, the stored-key count, and the dependence structure. A finer marginal-rate partition does not restore tuple independence: symmetry enforced by a canonical source has few rate blocks but perfect dependence within each orbit. Model order counts rate blocks; it says nothing about the joint law, and the query-level consequences of shared sources are the subject of the next section.

12.4 Query Error Contracts

Tuple rates do not determine a query’s error. The lineage fixes which sources a query reads, and the exact truth profile fixes which of them can err in each direction. Since no prior over exact databases is assumed, the natural guarantee is a contract: a bound valid for every exact truth profile with the given answer. It uses the same lineage as exact probability evaluation for one profile, but answers a different question.

Fix a nonconstant monotone Boolean function \(F : \{0, 1\}^n \to \{0, 1\}\) on distinct sources (the lineage after aliases are identified). In the one-sided false-positive model each source reports \(Y_i^{+}(x) = x_i \lor E_i\) with independent \(E_i \sim \mathrm {Bernoulli}(\fprate _i)\); in the separate one-sided false-negative model it reports \(Y_i^{-}(x) = x_i \land \lnot E_i'\) with independent \(E_i' \sim \mathrm {Bernoulli}(\fnrate _i)\). The contracts are \[ \mathcal {E}_0(F) = \max _{x : F(x) = 0} \Prob {F(Y^{+}(x)) = 1}, \qquad \mathcal {E}_1(F) = \max _{x : F(x) = 1} \Prob {F(Y^{-}(x)) = 0}. \] Let \(\mathcal {B}_0\) be the inclusion-maximal false inputs and \(\mathcal {B}_1\) the inclusion-minimal true inputs, and let the one-sided sensitivities be \(s_b(F) = \max _{x : F(x) = b} \Card {\{i : F(x \oplus e_i) \ne F(x)\}}\). For monotone functions sensitivity equals certificate complexity, a fact due to Nisan [Nis91].

These identities are elementary consequences of monotone boundary certificates, not a new inference algorithm; sensitivity-based noise bounds for Boolean functions are established [Gop+16]. In a two-sided channel the same expressions are upper bounds, not equalities: for \(F = X_1 \land X_2\) with \(\fprate \le 1 - \fnrate \) the worst two-sided false-positive rate is \(\fprate (1 - \fnrate )\), while the one-sided formula gives \(\fprate \). This is the query-level counterpart of the worst-case rate bounds of Proposition 3.3.2.

Shared construction randomness, such as a Bloom filter’s occupancy, is a structured middle ground between the two theorems.

The conditional model allows marginal dependence (two absent sources fire together with probability \(\Expect {P^2} \ge (\Expect {P})^2\)), but it is one specific dependence model and does not stand in for arbitrary correlation.

A tractable class: binary witnesses

Write the lineage in normalized positive disjunctive normal form \[ F = \bigvee _j \bigwedge _{i \in S_j} X_i, \] with aliases identified, duplicates removed, and absorbed witnesses dropped. Then \(s_1 = \max _j |S_j|\), while \(s_0\) is the size of the largest inclusion-minimal hitting set of the witness family, not of the minimum one. For binary witnesses, let \(G\) be the graph whose edges are the supports \(S_j\): false inputs are independent sets, maximal false inputs are maximal independent sets, and \(s_0(F_G) = |V(G)| - i(G)\), with \(i(G)\) the independent domination number. Computing \(i(G)\) is hard already on restricted cubic graphs [LPL15], so limiting witnesses to size two is not enough.

For a target query false-positive rate \(\delta \in (0, 1)\), homogeneous independent one-sided sources sized as \begin {equation} \label {eq:query-calibration} \fprate \le 1 - (1 - \delta )^{1/s_0} \end {equation} suffice when \(s_0 > 0\); under arbitrary dependence \(\fprate \le \delta / s_0\) suffices instead. The 12-cycle of binary witnesses shows why the contract, not a convenient test profile, is the right target. It has \(s_0 = 8\) and \(s_1 = 2\), so a per-source rate of \(0.05\) permits a worst-case query false-positive rate of \(1 - 0.95^8 \approx 0.337\), attained at the false profile with true sources at positions \(\{0, 3, 6, 9\}\) and all others false. The all-false profile, which needs two adjacent false positives, errs far less often, so validating a structure on the all-false profile alone does not validate the contract.

A stored-filter calibration

The relations manuscript tests the contract on a concrete structure [Tow26h, Sec. 7]. The structure is a frozen partitioned Bloom filter representing a symmetric relation: \(k\) partitions of \(m\) bits, one hash per partition, queried from the serialized bit array alone. Canonical mode stores unordered keys (Theorem 12.3.7); ordered mode stores both orientations and ORs the two probes (Theorem 12.3.3). Both are symmetric and FNR-zero by construction. Under ideal independent hashing and conditional on the hashes of the inserted keys, an absent key tests positive with probability \(p(C) = \prod _{j=1}^{k} o_j / m\), where \(o_j\) is the occupied count of partition \(j\), which is exactly the shared-rate setting of Corollary 12.4.2.1. Construction retries salts until \(p(C) \le \fprate \), an acceptance test that inspects inserted-key hashes only.

The workload is the 12-cycle query, \(128\) fixed bundles for each of three profiles (all sources absent; the extremal false profile above; a true profile with an adjacent pair) over an exact relation of \(2{,}816\) member orbits, with query target \(\delta = 0.05\). Three sizings were built \(160\) times each, with independent seeds: canonical storage sized per tuple (\(\fprate = \delta \)); canonical storage sized by (12.1) (\(\fprate = 1 - 0.95^{1/8}\)); and ordered storage with OR repair, sized for its \(16\) directed sources (\(\fprate = 1 - 0.95^{1/16}\), \(5{,}632\) ordered keys). Table 12.1 reports the results.

Sizing each source at the tuple target passes an all-false check against \(0.05\) (measured \(0.02646\)) while its boundary-profile error is about \(0.329\); calibrated sizing restores about \(0.05\) in both representations. The mean ideal-model predictions (\(0.33224\), \(0.04900\), \(0.04919\)) fall inside the measured intervals, and an exact rational audit of the recorded occupancies confirms that all \(320\) calibrated builds meet the ideal-model target. The ordered method’s observed mean slightly exceeds \(0.05\) while its interval covers the prediction: the experiment demonstrates compatibility with the ideal model, not a certificate that the concrete keyed hash (BLAKE2b) attains it. At equal ideal query targets, canonical storage used \(55.8\%\) fewer bytes than ordered storage, mainly from representative deduplication and the rate cost of OR repair. The sharp contract itself buys little here: sizing by the conservative union bound (\(\fprate = \delta / 8\)) would allocate \(3{,}768\) bytes against \(3{,}752\), \(16\) bytes or \(0.43\%\) more. The limits are equally explicit: one synthetic query family, one size and target, one hash implementation, and one-sided errors.

Rates preserved, inference cost changed

Canonical symmetry preserves every tuple rate yet can change the cost of exact query inference. For a simple cubic graph \(G\), let an exact relation \(D(v, a, b, c)\) list each vertex’s three neighbors and consider \(\exists v, a, b, c : D(v, a, b, c) \land R(v, a) \land R(v, b) \land R(v, c)\) with every queried \(R\) tuple an exact nonmember of rate \(p\). With independent directed sources the query false-positive rate is \(1 - (1 - p^3)^{|V|}\). Once symmetry identifies opposite orientations, each undirected edge is one bit used twice; each tuple still has rate \(p\) and each witness probability \(p^3\), but the query is false exactly when the zero edges form an edge cover, so at \(p = 1/2\) its false-positive rate is \(1 - \#\mathrm {EdgeCovers}(G)/2^{|E|}\). Counting edge covers is #P-hard on planar cubic graphs [CLX12], and the refinement to simple graphs is stated by Amarilli and Monet [AM22]; the example is an application of that known hardness. A compact worst-case contract and a fixed-profile exact probability are different objectives, and dissociation bounds [GS14] are the established tool for the second when sources repeat.

12.5 Approximate Joins in Databases

Section 12.2 treated the join algebraically. This section reads the database similarity join in the same vocabulary: the standard MinHash-LSH analysis supplies the rates of a sketch-based approximate relation, and the chapter’s main caution, that tests reading the same randomness are not independent, applies inside the pipeline itself.

The similarity join

For relations \(R_1, R_2\) whose rows are token sets, the similarity join at threshold \(\theta \) is \(R_1 \bowtie _\theta R_2 = \{(r_1, r_2) : J(r_1, r_2) \ge \theta \}\), the cross-relation pairs whose Jaccard overlap is at least \(\theta \). Record linkage, near-duplicate detection, and entity resolution reduce to it. The naive evaluation computes \(|R_1| \cdot |R_2|\) Jaccard values; the sketch evaluation is an approximate relation \(\tilde S_\theta (r_1, r_2) = [\hat J(r_1, r_2) \ge \theta ]\).

The MinHash sketch (Definition 10.4.1) supplies \(\hat J\): the fraction of agreeing signature coordinates is an unbiased estimate of \(J\) (Proposition 10.4.1). With length-\(k\) signatures, a false positive needs \(\hat J\) to overshoot a sub-threshold truth and a false negative needs it to undershoot a super-threshold one, and the Hoeffding bound of Proposition 10.4.2 gives \(\Prob {|\hat J - J| \ge t} \le 2 \exp (-2kt^2)\) with \(t\) the gap to \(\theta \), so both rates fall exponentially in \(k\) away from the threshold. These are pointwise rates for a fixed pair; pairs sharing a row share that row’s signature and are not independent, exactly as join outputs sharing an operand tuple are not (Theorem 12.2.4).

LSH bands

The remaining quadratic factor is candidate enumeration, which locality-sensitive hashing (Section 10.4) prunes. Partition the signature into \(b\) bands of \(r\) rows (\(k = br\)), hash each band, and call \((r_1, r_2)\) a candidate if any band collides. A band collides with probability \(J^r\), so a pair survives with probability \(1 - (1 - J^r)^b\), an S-curve with transition near \(J^* \approx (1/b)^{1/r}\). The pipeline reports a pair when it is a candidate and passes the threshold test. Both tests read the same signature, so they are dependent and the product rules of Theorem 12.2.3 and chapter 3 do not apply. Sound statements remain: the pipeline’s false-positive rate is at most that of either stage, and its false-negative rate is at most the sum of the two stages’ false-negative rates, by the union bound, whatever the dependence.

Worked example

Take \(|R_1| = |R_2| = 1000\) (\(10^6\) candidate pairs), \(\theta = 0.5\), and \(k = 200\) as \(b = 20\) bands of \(r = 10\). The threshold test is sharp: a pair at \(J = 0.3\) is emitted with probability at most \(\exp (-2 \cdot 200 \cdot 0.2^2) = e^{-16} \approx 1.1 \times 10^{-7}\), a pair at \(J = 0.7\) is missed with the same bound, and at \(J = \theta \) the test is a coin flip.

The banding dominates, and its tuning needs care. The transition sits at \(J^* = (1/20)^{1/10} \approx 0.74\), above the threshold, an aggressive precision-favoring choice: the survival probability \(1 - (1 - J^{10})^{20}\) is about \(0.436\) at \(J = 0.7\), \(0.0194\) at \(J = 0.5\), and \(1.18 \times 10^{-4}\) at \(J = 0.3\). Suppose \(0.1\%\) of pairs have \(J \ge 0.5\), \(1\%\) lie in \([0.3, 0.5)\), and the rest lie below \(0.3\), and take \(J = 0.7\), \(0.5\), and \(0.3\) as representative of the three bands (the last two are upper ends, so their counts are upper estimates). The candidate set is then about \[ 1000 \cdot 0.436 + 10000 \cdot 0.0194 + 989000 \cdot 1.18 \times 10^{-4} \approx 436 + 194 + 117 \approx 747 \] pairs, three orders of magnitude below the all-pairs scan, verified with about \(1.5 \times 10^5\) signature comparisons against \(2 \times 10^8\) for the naive scan. The cost is recall: fewer than half of the strongly similar pairs and about \(2\%\) of threshold pairs survive. Centering \(J^*\) on \(\theta \) (the standard tuning \((1/b)^{1/r} = \theta \), here with more bands) admits more candidates in exchange for higher recall. This precision and recall trade through \((b, r)\) is the standard operating regime of MinHash-LSH similarity joins. Section 12.6 closes the chapter.

12.6 Bridge

The chapter read the approximate relation through its operators, its invariants, and its queries, and one distinction carried all three: a tuple’s rate is a property of one probe, while a query’s error depends on which stored outcomes its probes share. Section 12.1 separated exact truth, construction randomness, and source identity. Section 12.2 found selection rate-transparent, projection amplifying false positives with rates indexed by the fiber profile \((k, j)\), and the join correct per tuple but correlating outputs that share an operand tuple; composing operators requires the lineage over sources, not a chain of per-operator rate updates. Section 12.3 showed every invariant fragile under independent tuples, and showed that the constructions which enforce one (canonical sources, OR-closure, deletion) move error into the stored-key count and the dependence structure rather than removing it. Section 12.4 turned the source view into truth-uniform contracts, \(1 - (1 - \fprate )^{s_0}\) and \(1 - (1 - \fnrate )^{s_1}\) under independent one-sided sources, and showed in a stored-filter experiment that sizing sources at the tuple target can miss a query target by a factor of more than six, while calibrated sizing meets it. Section 12.5 read the MinHash-LSH similarity join in the same terms.

Chapter 13 closes Part III with the type algebra that organizes approximate sets, functions, relations, and sketches as \(\Bernoulli {T}\) constructors. An approximate relation is \(\Bernoulli {\BoolType }\) on a product universe, and the type view supplies its marginal rates. The lesson of this chapter travels with it: marginals compose cleanly, but joint behavior is fixed by shared sources, which a type signature alone does not record.

12.7 Notes and Lineage

BF Lineage. The relational model and its operators are due to Codd [Cod70]. A probabilistic relational algebra, whose tuples carry event expressions evaluated under independence of basic events, goes back to Fuhr and Rölleke [FR97]; tuple-independent query evaluation [DS07], its dichotomy [DS12], and the survey of Suciu et al. [Suc+11] followed, with Trio and ULDB [Wid05; Ben+08] and MayBMS [Hua+09] as systems. Boolean lineage and provenance semirings [GKT07] already record how tuples contribute to an answer; once a truth profile and a construction law are fixed, every query probability in this chapter is an instance of probabilistic inference over such lineage, and the fact that the uncertainty originates in a compact representation rather than in the data does not evade that prior art. Conditioning probabilistic databases [KO08] and dissociation bounds for repeated sources [GS14] are the established tools for the two situations Section 12.4 touches. The contracts themselves are elementary consequences of monotone certificates (sensitivity equals certificate complexity for monotone functions, Nisan [Nis91]; sensitivity-based noise bounds, Gopalan et al. [Gop+16]). Constraint repair is the subject of consistent query answering [Ber11]. Approximating a relation with a Bloom filter [Blo70] in query processing goes back at least to Bloom-filter semijoins [Mul90], and the similarity join combines MinHash [Bro97] with locality-sensitive hashing [IM98], systematized as a set-similarity operator by Sarawagi and Kirpal [SK04].

BF What this chapter adds. The chapter is a re-synchronized presentation of Towell [Tow26h], revised in October 2026 after an audit retracted several earlier claims. It contributes explicit source semantics in the book’s notation, audited operator and invariant formulas, the separation of marginal rates from shared-source dependence, and a reproducible calibration study whose finding (tuple-sized sources at \(0.05\) give a boundary-profile query error near \(0.33\); calibrated sizing restores about \(0.05\)) is a design lesson rather than a new theorem. A targeted search did not locate the identical query-calibration formulation, which does not establish originality.

BF Left out. Negation, aggregates, adaptive query selection, approximate attribute comparisons, weighted model counting beyond the path and cycle class, construction-specific guarantees for concrete hash families, and any confidentiality claim (see the scope remark in Section 12.6).

Chapter 13
The Algebra of Approximate Types

13.1 Algebraic Types Recap

Twelve chapters in, the same calculations keep recurring. The Kronecker product of channel matrices that organized \(m\) Boolean queries in chapter 4 returned for the n-ary channels of chapter 9 and the relational products of chapter 12; the set-composition rules of chapter 6 lined up with chapter 9’s; CountMin and HyperLogLog were noisy counters, a different output type under the same two axioms. The thread is that each structure is a \(\Bernoulli {T}\) for some algebraic type \(T\). This chapter organizes the structures by type. The organization has direct predecessors in approximate and probabilistic programming, where type qualifiers carry a correctness probability through a program (see the Notes and Lineage at the end of the chapter); what follows records which constructors make the error law of a composite value factor, under which independence hypotheses, and where factorization fails.

BF The four constructors. Algebraic data types are built from four constructors: the product \(T \times U\) (pairs, records, structs; cardinality \(|T||U|\)), the sum \(T + U\) (tagged unions, variants, enums; cardinality \(|T| + |U|\), with a tag recording which summand), the list \(L T\) (finite sequences of any length), and the function \(T \to U\) (the most general, into which the others embed). They close under composition, so any type built from primitives and these four has a clear structural decomposition.

Each approximate structure so far lives at one constructor. An approximate set (Chapter 5) is a noisy predicate \(\Bernoulli {T \to \BoolType }\), with the chapter-6 composition rules (Chapter 6) the rules for composing such predicates. An approximate map or channel (Chapter 9) is a \(\Bernoulli {T \to U}\) for finite \(T, U\), the n-ary channel. A sketch (Chapter 10) is a noisy value of an aggregate type, CountMin and HyperLogLog as \(\Bernoulli {\mathbb {N}}\), MinHash as \(\Bernoulli {[0,1]}\). An approximate relation (Chapter 12) is a noisy predicate on a product, \(\Bernoulli {T \times U \to \BoolType }\). The type tells how the joint error law of a composite value factors when the components’ noise is independent; it does not by itself give the error of a computation on such values (Section 13.3).

BF The lift. Write \(\Bernoulli {-}\) as a type-level operation: \(\Bernoulli {T}\) is the type of random approximate values over \(T\), with the atom \(\Bernoulli {\BoolType }\) the noisy bit of Part I. The substantive content of the chapter is how \(\Bernoulli {-}\) behaves on each constructor. Section 13.2 shows it factors through three of the four when the relevant noise is independent: the product (a tensor of factor lifts), the list (a per-position lift), and the function (a per-input lift), each by the Kronecker factorization of chapters 4 and 9. The sum is the exception: a sum value’s tag can itself err, coupling the two components, and there the factorization breaks even with independent noise. Section 13.3 collects the rules, states what they do and do not give, and identifies the sum’s tag error with the membership error of approximate sets.

13.2 \(\Bernoulli {T}\) for Composite \(T\)

This section gives the lift \(\Bernoulli {-}\) on each constructor. Three of the four factor when the relevant noise is independent; the sum is the exception.

Product types: the Kronecker isomorphism

A noisy pair is a pair of noisy values. If \(\tilde t\) has channel \(Q_T\) and \(\tilde u\) has channel \(Q_U\), with independent noise, the joint channel at each \((t, u)\) is the product of the marginals, so the joint matrix is the Kronecker product \(Q_T \otimes Q_U\), the chapter-4 and chapter-9 factorization (Theorem 4.4.1, Theorem 9.3.1) read at the type level.

Independence of the two components’ noise, the hypothesis of the theorem and the type-level analogue of Axiom 1, factors the joint into a product across components, and Axiom 2 keeps each row stable, so the proof is the chapter-9 calculation with the two factors now of different dimensions. The parameter count is additive: without independence the joint would need \(K_T K_U(L_T L_U - 1)\) parameters, but under it only \(K_T(L_T - 1) + K_U(L_U - 1)\), the parameters of \(Q_T\) plus those of \(Q_U\). Products are the well-behaved case, where approximation distributes and parameters grow additively.

Sum types: the tag error breaks the factorization

A sum \(T + U\) tags each value \(\mathrm {Left}(t)\) or \(\mathrm {Right}(u)\). An approximate sum value errs in two ways: a tag error flips the tag with probability \(\delta \), and a payload error corrupts the value within the correct component at rate \(\fprate _T\) or \(\fprate _U\). For a latent \(\mathrm {Left}(t)\), \begin {gather*} \Prob {\widetilde {\mathrm {Left}(t)}} = (1 - \delta )(1 - \fprate _T), \qquad \Prob {\mathrm {Left}(t'\!\neq t)} = (1 - \delta )\tfrac {\fprate _T}{|T| - 1}, \\ \Prob {\mathrm {Right}(\cdot )} = \delta , \end {gather*} the last spread over the wrong component.

The reason is structural and is the heart of the product-versus-sum distinction. In a product, both components are always active and independent, so the channel tensors. In a sum, the tag error couples the components: when the tag is correct only one component’s payload channel applies, and when it is wrong the value is drawn from the other component entirely. The block-diagonal form \(\Bernoulli {T} \oplus \Bernoulli {U}\) (one payload channel per summand, zero off-diagonal) is only the \(\delta = 0\) degenerate case, a sum with an exact tag; any tag noise puts mass in the off-diagonal blocks and no factorization recovers it. This is Core Principle 2: products preserve parametric parsimony, sums break it.

The break is not a peripheral failure. The optional type \(\mathrm {Maybe}(T) = T + \Unit \) adds a “nothing” case, and its tag error is approximate set membership: a tag flip on \(\mathrm {Just}(t)\) produces \(\mathrm {Nothing}\), a false negative, and a flip on \(\mathrm {Nothing}\) produces some \(\mathrm {Just}(t)\), a false positive, with \(\delta = \fprate \) for negatives and \(\delta = \fnrate \) for positives. The sum’s tag error is exactly the atomic membership error the whole framework rests on, which is why the sum constructor cannot be made to factor: factoring it away would factor away the framework’s irreducible rate.

List and function types

A list of known length \(n\) is the product \(T^n\), so \(\Bernoulli {T^n} \cong \Bernoulli {T}^{\otimes n}\), the Kronecker power of the per-position channel, with parameter count \(K_T(L_T - 1)\) independent of \(n\). A variable-length list, however, is a sum over lengths, \(L T \cong \sum _{n \ge 0} T^n\), so a noisy length is a sum-type tag error over the length index: \(\Bernoulli {L T}\) is a mixture over the noisy length (a \(\Bernoulli {\mathbb {N}}\)), each component a per-position Kronecker product, and it does not reduce to a single clean product. A HyperLogLog cardinality estimate plays the role of this noisy length, and it combines with per-element noise by the mixture rule, not by a simple per-element product.

A noisily approximated function \(f : T \to U\) produces one \(\Bernoulli {U}\) per input, so \(\Bernoulli {T \to U} \cong T \to \Bernoulli {U} \cong \prod _{t \in T} \Bernoulli {U}\), the chapter-9 framework: the channel is a function from input blocks to output distributions. Under Axiom 1 across inputs this factors over inputs as a Kronecker product of \(|T|\) per-input channels, with parameter count \(|T|(L_U - 1)\), growing with the domain. Approximate sets (\(U = \BoolType \)) and relations (\(T\) a product) are sub-cases. Structures whose answers share stored state, such as a CountMin sketch read as \(\Bernoulli {T \to \mathbb {N}}\), satisfy the per-input marginal form but not the Kronecker product across inputs (Section 10.1).

The first rows hold under the independence hypotheses in the last column; the last row records where factorization fails even when the noise is independent. Section 13.3 collects the rules.

Worked example

A 3-class classifier reporting a class label and a times-seen count has output type \(\{\text {cat}, \text {dog}, \text {bird}\} \times \mathbb {N}\), a product of a 3-element enumeration (governed by a \(3 \times 3\) confusion matrix \(Q_{\text {class}}\)) and a count (a CountMin-style \(\Bernoulli {\mathbb {N}}\), \(Q_{\text {count}}\)). Under independence the joint factors as \(\Bernoulli {\{\text {cat}, \text {dog}, \text {bird}\}} \otimes \Bernoulli {\mathbb {N}}\), the Kronecker product \(Q_{\text {class}} \otimes Q_{\text {count}}\), with additive parameter count \(3 \cdot 2 + K_{\mathbb {N}} (L_{\mathbb {N}} - 1)\). The factorization describes the joint law of the reported pair. It does not by itself give the error of a computation on such pairs. A test such as “the label is cat and the count exceeds \(10\)” is a Boolean formula over two noisy components, and its error depends on the latent pair, not only on whether the test is truly satisfied: a true dog seen \(12\) times and a true cat seen \(3\) times are both negatives, with different false-positive probabilities. Exact propagation is per latent assignment (Proposition 3.3.1), and a single rate pair for the output exists only as a prior average or a worst-case bound (Example 3.1, Proposition 3.3.2). The type structure says which factors are independent; the computation still has to be analyzed.

13.3 Structure Preservation and Its Limits

Section 13.2 gave the lift on each constructor. This section collects the rules, states what they give, and marks where they stop.

The rules as one map

Under the independence hypotheses of the table in Section 13.2, the lift \(\Bernoulli {-}\) takes a type-level operation to a channel-level one. The product becomes a Kronecker product (Theorem 13.2.1), the list a Kronecker power, the function a per-input Kronecker product. A map that takes operations to corresponding operations is structure-preserving; the categorical name is functor, and the Kronecker product is the tensor of the monoidal category of finite stochastic matrices. In summary:

Within their hypotheses the first three rows are isomorphisms: the type and the channel specify the same \(\Bernoulli {-}\) value up to relabeling. The fourth row is the exception, the sum, where the tag error couples the components and no factorization recovers the channel (Proposition 13.2.1); the mixture in the list row arises when the length itself is noisy. That exception is where the framework’s atom sits: the optional type’s tag error is the membership error of an approximate set (§13.2).

What the table gives, and what it does not

The table states how the joint error law of a composite value factors when its components’ noise is independent, and how many parameters that law needs. That is useful bookkeeping: a structure built from the three factoring constructors with independent noise has a joint law fixed by its components’ channels, and the sum marks where a tag error couples components even then.

It is not a composition calculus for computations on approximate values, for three reasons the earlier chapters make concrete. First, an operation applied to noisy components produces an output whose error depends on the latent input, not only on the output’s truth value: the AND of two noisy bits already has three distinct negative-case rates, so a two-rate pair does not propagate exactly through a program (Example 3.1). Exact propagation is per latent assignment (Proposition 3.3.1), and a single pair survives only as a prior average or a worst-case bound (Proposition 3.3.2). Second, programs that use a value twice, and stages that are stored rather than re-randomized, break the independence the table assumes (Remark 9.3.1). Third, some structures do not meet the function row’s hypothesis at all: a CountMin sketch read as \(\Bernoulli {T \to \mathbb {N}}\) answers different items from shared counters, so its per-item errors are dependent and the per-input Kronecker product does not describe them (Section 10.1).

With those limits stated, the earlier structures fit the vocabulary as follows. An approximate set (Chapter 6) is \(\Bernoulli {T \to \BoolType }\), its Axiom 1 being the function row’s hypothesis, and the chapter-6 operation rates are per-latent-assignment calculations of the kind Proposition 3.3.1 makes general. An approximate relation (Chapter 12) is \(\Bernoulli {T \times U \to \BoolType }\), a predicate on a product domain, whose query errors chapter 12 bounds query by query rather than reading them off the table. Sketches (Chapter 10) are noisy values of aggregate types whose per-query marginals fit the table and whose joint laws do not. The n-ary channel of Chapter 9 is the building block \(X \to Y\) underneath them all.

One object, several readings

There is one underlying object across the book. The atom is the noisy bit \(\Bernoulli {\BoolType }\), and the other structures are \(\Bernoulli {T}\) values for algebraic types built from \(\BoolType \) and the constructors. The type algebra organizes where independence makes their error laws factor and where it does not; the error of computations over them is analyzed per latent assignment, or bounded. The remaining chapters use this organization rather than extend it: chapter 14 analyzes a seed-search construction near the entropy floor, chapter 15 the perfect hash filters, chapter 16 catalogues where the independence assumptions break, and chapters 17 and 18 turn the same rates to cryptographic use.

13.4 Closing Part III

Three parts are now in place. Part I built the atom: the noisy bit, a \(\Bernoulli {\BoolType }\) value with a \(2 \times 2\) channel and rates \((\fprate , \fnrate )\). Part II specialized it one direction, to predicates on a universe, and worked out the algebra of approximate sets, their composition (chapter 6), classification measures (chapter 7), and information-theoretic floor (chapter 8). Part III specialized it a second direction, to maps and types: Chapter 9 lifted the binary channel to \(K \times L\) channels, Chapter 10 read streaming counters and estimators as such channels, chapter 11 placed randomized deciders on the same rate plane, Chapter 12 turned to predicates on pairs, and this chapter organized the four by type.

Each part widens the type structure. Part I fixes the codomain to \(\BoolType \) and studies the atom; Part II keeps the codomain \(\BoolType \) and studies functions \(T \to \BoolType \); Part III releases both domain and codomain to range over the algebraic types built from \(\BoolType \) and the four constructors. The two rates of Part I become the channel matrix of Part III. Under independent noise, products go to Kronecker products, lists to per-position lifts, functions to per-input lifts; the sum is the exception, where the tag error couples the components, the same tag error that is the membership error of an approximate set. Part III also recorded two limits that the rest of the book keeps in view: rates do not propagate as a single pair through computations, and stored stages or shared inputs break the independence that factorization needs.

Three parts remain. Part IV turns to constructions near the chapter-8 floor, a bound that is classical and is met to within lower-order terms by known retrieval-based filters (chapter 14’s notes give the lineage). Chapter 14 analyzes the Bernoulli Hash Function, a seed-search construction that addresses a fingerprint table through a perfect hash, and Chapter 15 the perfect hash filters built from minimal perfect hashing. Part V draws the boundary and turns the algebra to cryptographic use: Chapter 16 catalogues where the independent-error model fails (correlated errors, drifting rates, adversarial queries), and chapters 17 and 18 read the algebra in reverse, the salt of a Bernoulli hash function as a key and the acceptance predicate as a cipher map. Chapter 19 then closes the book by re-reading the framework in operator form, elements as basis vectors and noise as a stochastic matrix, where the Kronecker products of Section 13.3 become the tensor factorization of an independent set’s noise.

13.5 Notes and Lineage

BF Lineage. Values that are correct with a tracked probability, composed through programs, have a direct predecessor in approximate programming. Decaf [Bos+15] gives each value a type qualifier @Approx(p), the probability that it is correct, and infers the qualifier of an expression as the product \(p_a\, p_b\, p_{\mathrm {op}}\) of its operands’ and operator’s probabilities; Rely [CMR13] verifies such reliability bounds for programs on unreliable hardware, Chisel [Mis+14] optimizes against them, EnerJ [Sam+11] separates approximate from precise data by type, and \(\mathtt {Uncertain}\langle T\rangle \) [BMM14] makes a distribution over \(T\) a first-class value. The probability monad behind Remark 13.3.1 is Giry [Gir82]; its use as a semantics of probabilistic programs is Kozen [Koz81] and Ramsey and Pfeffer [RP02], with Wadler [Wad92] for monads in programming and Mac Lane [Mac98] for the categorical machinery. The four-constructor view of algebraic data types is standard [Hud89].

BF What this chapter adds. The constructor table follows the data-type manuscript Towell [Tow26k]. Relative to Decaf’s single correctness probability, the lift tracks a confusion matrix per type, so the false-positive and false-negative directions stay separate, and the chapter records which constructors make a composite value’s error law factor (products, lists, and functions under independence) and which do not (sums, through the tag error, identified with approximate-set membership error). It also corrects earlier statements of the table as a composition calculus: the table describes values, not computations, and two-rate pairs do not compose exactly through programs.

BF Left out. A type system. A sound two-rate (confusion-matrix) qualifier discipline over sum, product, and function types, with inference and with the per-latent-assignment and dependence issues above handled soundly, would extend Decaf’s single-probability qualifiers; it is an open research direction, not a result of this book. Recursive types beyond lists, higher-order functions, and dependence tracking across shared subterms are also left out.

Part IV
Optimality and Construction

Chapter 14
Optimal Bernoulli Sets (BHF)

14.1 The Optimality Question

Chapter 8 closed Part II with a contrast it could not resolve. The entropy floor (Theorem 8.3.1) certified that some FNR-zero representation could in principle reach \(-\log _2 \fprate \) bits per element, but it exhibited none; the standard Bloom filter, the only concrete FNR-zero scheme on the table, sat a fixed multiplicative factor above that floor. The bound was diagnostic, not constructive. This chapter supplies constructions: an explicit FNR-zero approximate set that meets the floor up to lower-order terms at an exponential build cost, and a practical one that replaces the Bloom filter’s multiplicative overhead with an additive constant.

The gap

Theorem 8.3.1 of Section 8.3 states that any FNR-zero approximate-set representation with false-positive rate at most \(\fprate \) requires \[ L / n \;\ge \; -\log _2 \fprate \;+\; o(1) \] bits per element in the asymptotic regime \(N \gg n\), \(\fprate \) fixed. The argument is a counting bound: \(2^L\) representations must cover \(\binom {N}{n}\) possible target sets, and each representation describes at most \(\binom {|A_R|}{n}\) true sets inside its accepted set, forcing \(L\) to grow linearly in \(n \log _2(1/\fprate )\).

Proposition 8.4.1 of Section 8.4 computes the Bloom filter’s budget from Definition 5.2.1: at the optimal hash count \(k^* = (m/n)\ln 2\), the per-element cost is \[ m / n \;=\; \log _2(1/\fprate ) \cdot \log _2 e, \] or \(\log _2 e \approx 1.4427\) times the floor, a constant independent of \(\fprate \). The overhead traces to a structural choice: \(k\) independent hashes write into a flat shared bit array, and the analysis maximizes per-bit entropy by tuning the array to half occupancy. Half-occupancy is the wrong target. Bits should pay for the discrimination the structure performs, not for the encoding’s internal balance.

The constructive question

Section 8.4 isolated three Bloom design choices, \(k\) independent hashes, a flat shared bit array, and a conjunction query, and observed that closing the gap requires abandoning at least one. A construction that answers the question needs five properties: FNR-zero by construction, so it stays on the perfect-filter axis with the Bloom filter; an FPR provably at most \(\fprate \); a per-element budget matching \(-\log _2 \fprate \) up to a lower-order correction; a query running in time comparable to Bloom’s; and a build that finishes in time polynomial in \(n\). The floor, which goes back to Carter et al. [Car+78], bounds the third property from below. The question is which explicit constructions meet which of the five, and at what price.

The answer in sketch

Two constructions answer it from opposite ends, and the chapter treats both.

The Bernoulli hash function (BHF) abandons stored per-element structure altogether. It fixes an acceptance set of density \(\fprate \) in the output range of a hash function and searches for a single salt under which every member of \(A\) hashes into the acceptance set; the salt is the whole representation, and a query hashes \(x\) with the salt and tests acceptance. Each member is accepted with probability \(\fprate \) under a given salt, so a salt succeeds with probability \(\fprate ^n\), and the first success has an index whose binary length is \(n \log _2(1/\fprate ) + O(1)\) in expectation. The BHF therefore meets the floor up to lower-order terms, with no integer rounding, and its query is a single hash evaluation. What it gives up is the fifth property: the expected number of salts tried is \(\fprate ^{-n}\), exponential in \(n\). This is single-seed search, the mechanism that modern perfect-hashing constructions apply to small buckets [EGV20; LSW24; Leh+25].

The perfect hash filter keeps the build polynomial. A minimal perfect hash \(\phi : A \to [n]\) addresses a table of \(r\)-bit fingerprints with \(r = \lceil -\log _2 \fprate \rceil \), and a query compares the stored fingerprint with the query’s own. It meets every property except the third exactly: the fingerprints cost \(r\) bits per element, but a minimal perfect hash cannot cost less than \(\log _2 e \approx 1.44\) bits per element [FK84], so the total sits a constant additive amount above the floor, against the Bloom filter’s multiplicative factor. The pairing of a minimal perfect hash with fingerprints, and its \(\log _2 e\) overhead, are described by Dietzfelbinger and Pagh [DP08, Sec. 1]. Retrieval structures close the remaining gap to a factor \(1 + \delta \) at linear build time (Section 14.3).

Section 14.2 gives the BHF in full, Section 14.3 its expected size and the perfect hash filter’s budget, Section 14.4 the false-positive analysis of both, and Section 14.5 the build costs that separate them.

14.2 The BHF Construction

The Bernoulli hash function stores no table. It stores one salt, chosen so that a fixed acceptance rule happens to accept every member of the designated set, and it answers a query by applying the same rule to the query. This section states the construction, proves that it has no false negatives, computes the probability that a candidate salt succeeds, and works one example small enough to build. Section 14.3 computes the expected size, and Section 14.5 the build cost that the size is bought with.

Acceptance predicate and salt

Fix a finite universe \(\Universe \), a designated set \(A \subseteq \Universe \) with \(|A| = n\), a modulus \(N \ge 1\), and a threshold \(t \in \{0, 1, \ldots , N - 1\}\). Let \(h : \Universe \times \{0, 1\}^* \to \{0, 1, \ldots , N - 1\}\) be a hash of an element and a bit-string salt, modeled as a random oracle: its values at distinct pairs \((x, b)\) are independent and uniform on \(\{0, \ldots , N - 1\}\). An element \(x\) is accepted under salt \(b\) when \begin {equation} \label {eq:bhf-accept} h(x, b) \;\le \; t, \end {equation} an event of probability \[ \fprate \;=\; \frac {t + 1}{N} \] for any fixed pair. The acceptance set \(\{0, \ldots , t\}\) is fixed before any salt is examined; it is part of the specification, not something read off the data. The admissible rates form the lattice \(\{j / N : 1 \le j \le N\}\), as fine as the modulus allows.

Enumerate bit strings in order of length and then lexicographically, \(b_1\) the empty string, \(b_2 = 0\), \(b_3 = 1\), \(b_4 = 00\), and so on, so that \(b_q\) has length \(\lfloor \log _2 q \rfloor \). A salt \(b\) is valid for \(A\) when every member is accepted under it.

The definition follows the source manuscript [Tow26j], which calls this the threshold predicate; its equality predicate, which lets the first member fix a target value \(h_0\) and requires the others to hit it, is the same construction with the acceptance set chosen after one hash and stored alongside the salt, and it costs the same number of bits in total. Algorithm 6 writes the construction out.

A failed salt is usually discarded early: the scan over \(A\) stops at the first rejected member, which arrives after an expected at most \(1/(1 - \fprate )\) hash evaluations. The query is one hash evaluation and one integer comparison, the cheapest query in Part IV.

The false-negative rate is exactly zero

Let \(x \in A\). The build returned \(b^{*}\) only because \(h(a, b^{*}) \le t\) held for every \(a \in A\), in particular for \(x\), and the query recomputes the same value \(h(x, b^{*})\). Hence \(\textsc {Query}(x) = \top \) for every member and every parameter setting, and \[ \fnrate \;=\; 0. \] The BHF sits on the FNR-zero arm of the perfect-filter line, with the Bloom filter. Its only failure mode is a non-member whose hash under \(b^{*}\) happens to fall in the acceptance set, at probability exactly \(\fprate \) (Proposition 14.4.1).

Success probability of a salt

Under the random-oracle model the \(n\) values \(h(a, b)\), \(a \in A\), are independent for a fixed salt, each in the acceptance set with probability \(\fprate \), so a candidate salt is valid with probability \begin {equation} \label {eq:bhf-success} s \;=\; \fprate ^{\,n}. \end {equation} Distinct salts give independent trials, so the index \(Q\) of the first valid salt is geometric with parameter \(s\): \(\Prob {Q = q} = (1 - s)^{q-1} s\) and \(\Expect {Q} = 1/s = \fprate ^{-n}\). Everything about the BHF’s cost follows from this one fact. Its size is the length of \(b_Q\), about \(\log _2 Q \approx n \log _2(1/\fprate )\) bits, which is the floor; its build is \(Q\) trials, about \(2^{n \log _2(1/\fprate )}\), which is exponential. Each bit of the representation is paid for with a doubling of the search.

The construction extends to maps without change of mechanism, which is why the manuscript treats sets as the zero-length-value case. For a map \(A \to \{0, 1\}^{*}\) whose values have encodings of mean length \(\mu \), a second hash of \((x, b)\) must also reproduce each member’s encoded value, so a salt is valid with probability \(\fprate ^{n} 2^{-n\mu }\) and the expected representation grows to \(n(\log _2(1/\fprate ) + \mu )\) bits, the space-accuracy scale for approximate maps (the maps themselves are the subject of Chapter 9). The rest of the chapter stays with sets.

A worked example

Take \(n = 8\) and \(\fprate = 1/4\) (\(N = 4\), \(t = 0\)). A salt is valid with probability \(s = 4^{-8} = 2^{-16}\), so the build tries \(2^{16} = 65{,}536\) salts on average, a fraction of a second. The returned salt has expected length \(\Expect {\lfloor \log _2 Q \rfloor } \approx 14.7\) bits (Theorem 14.3.1 gives the exact series), about \(1.83\) bits per element against the floor’s \(-\log _2(1/4) = 2\); the deficit is the constant slack of the theorem, which the few bits needed to record \(N\), \(t\), and the salt’s length more than repay. A Bloom filter at the same rate spends \(2 \log _2 e \approx 2.89\) bits per element. The same arithmetic at the chapter’s usual operating point, \(n = 1000\) and \(\fprate = 0.01\), gives \(s = 10^{-2000}\): the BHF’s size is still the floor, about \(6644\) bits, and its build is out of reach. Section 14.5 returns to what that means in practice.

14.3 Space Analysis

The BHF’s representation is its salt plus the two parameters \(N\) and \(t\). This section computes the salt’s expected length exactly and shows that it meets the floor of Theorem 8.3.1 to within an additive constant over the whole set, not per element. It then states the perfect hash filter, the practical structure that replaces search by addressing, and accounts for its budget honestly: fingerprints at the floor up to rounding, plus a perfect hash that cannot cost less than \(\log _2 e\) bits per element.

The expected salt length

The build scans salts in the order \(b_1, b_2, \ldots \) and stops at the first valid one, \(b_Q\), whose length is \(L = \lfloor \log _2 Q \rfloor \). Since \(L \ge \ell \) exactly when \(Q \ge 2^{\ell }\), and \(\Prob {Q \ge q} = (1 - s)^{q - 1}\) for the geometric index, \begin {equation} \label {eq:bhf-salt-series} \Expect {L} \;=\; \sum _{\ell \ge 1} \Prob {Q \ge 2^{\ell }} \;=\; \sum _{\ell \ge 1} (1 - s)^{2^{\ell } - 1}, \end {equation} with \(s = \fprate ^{n}\) from (14.2).

The slack is a constant for the whole set. At \(s = 2^{-16}\), the worked example of Section 14.2, the series (14.3) evaluates to \(14.67\), which is \(1.33\) bits below \(\ell ^{*} = 16\); at \(s = 2^{-20}\) it is \(18.67\), again \(1.33\) below. Nothing in the construction allocates bits per element, so the fractional part of \(\log _2(1/\fprate )\) is paid exactly: at \(\fprate = 0.01\) the BHF’s expected cost is at most \(6.644\) bits per element plus the \(O(\log n)/n\) parameter term, not the \(7\) that an integer-width fingerprint would take. The equality-predicate variant [Tow26j] has success probability \(\fprate ^{n-1}\) and stores its target value, \(\log _2(1/\fprate )\) bits, so its total is the same.

None of this is new as a statement about seed search. The expected index of the first of a sequence of independent trials, each succeeding with probability \(s\), has a binary length of \(\log _2(1/s) + O(1)\); this is the seed-index scale on which the bucketed seed searches of modern minimal perfect hashing operate [EGV20; LSW24], and Lehmann et al. [Leh+25, Sec. 1.1] analyze exactly the case of one seed required to satisfy independent Bernoulli constraints: near-optimal space and exponential construction. The BHF is that baseline, written in the vocabulary of this book. Its value here is as the construction that shows the floor of Theorem 8.3.1 is the right target, with the build cost made explicit rather than hidden.

The perfect hash filter

Polynomial build time comes from addressing instead of search. A minimal perfect hash function \(\phi : \Universe \to [n]\) is injective on \(A\) and maps it onto \([n] = \{1, \ldots , n\}\); it is data-dependent, constructed at build time from \(A\), and defined (arbitrarily) on non-members. Pair it with a fingerprint hash \(g : \Universe \to \{0, 1\}^{r}\), uniform on its codomain, of width \begin {equation} \label {eq:bhf-r} r \;=\; \lceil -\log _2 \fprate \rceil , \end {equation} the least integer with \(2^{-r} \le \fprate \), and fill a table \(F : [n] \to \{0, 1\}^r\) in one pass: \begin {equation} \label {eq:bhf-insert} \text {for each } a \in A: \quad F[\phi (a)] \;\coloneqq \; g(a). \end {equation}

A member reads back the fingerprint it wrote, so \(\fnrate = 0\); a non-member reads some member’s fingerprint and matches it with probability \(2^{-r} \le \fprate \) (Proposition 14.4.2). The structure is the standard static approximate-membership construction: the pairing of a minimal perfect hash with fingerprints of \(\lceil \log _2(1/\fprate ) \rceil \) bits, and its overhead of about \(\log _2 e\) bits per element, are described by Dietzfelbinger and Pagh [DP08, Sec. 1], who remark that the origin of the idea is unknown to them, and the program’s manuscript on the construction [Tow26m] restates it as a Bernoulli set. Chapter 15 builds on it.

The proposition is a lower bound on this design, not on approximate membership: the overhead is the cost of naming each member’s slot, which the floor does not require. It is additive, where the Bloom filter’s \(\log _2(1/\fprate ) \cdot \log _2 e\) (Proposition 8.4.1) exceeds the floor by a multiplicative factor, so the perfect hash filter wins once \(\fprate \) is small enough and loses at loose targets. Writing the addressing cost as \(c\) bits per element and ignoring the rounding of \(r\), it beats the Bloom filter exactly when \((\log _2 e - 1) \log _2(1/\fprate ) > c\), that is when \(\log _2(1/\fprate ) > c / 0.4427\). At the idealized \(c = \log _2 e\) the break-even is near \(\fprate = 2^{-3.26}\). Real minimal perfect hashes cost more: from about \(1.5\) bits per key for the seed-search constructions RecSplit and ShockHash, which spend build time to approach \(\log _2 e\) [EGV20; LSW24], through about \(2\) for hash-displace-and-compress [BBD09], to about \(3\) for BBHash [Lim+17]. The companion notebook (bernoulli-py/notebooks/ch14-bhf.ipynb) measures \(c = 3.23\) bits per key for its BBHash-backed filter at \(n = 1000\), which puts the break-even at \(\log _2(1/\fprate ) \approx 7.3\): at \(\fprate = 2^{-7}\) the Bloom filter is smaller (\(10.10\) bits per key against \(10.23\)), and from \(\fprate = 2^{-10}\) the perfect hash filter is (\(13.23\) against \(14.43\)). The perfect hash filter does not dominate the Bloom filter on space; it dominates at tight targets, by a margin that grows linearly in \(\log _2(1/\fprate )\).

Closing the gap at linear build time

The addressing cost disappears if the structure need not say where each member’s fingerprint is, only what the fingerprint of each member is. A retrieval structure (a static function) stores, for each \(a \in A\), an \(r\)-bit value \(v(a)\) retrievable as \(V(a)\), and returns arbitrary values on non-members; storing \(v = g\) and accepting \(x\) when \(V(x) = g(x)\) gives an FNR-zero filter with false-positive rate \(2^{-r}\) for non-members whose retrieved value is independent of \(g(x)\). Dietzfelbinger and Pagh [DP08] show that retrieval, and hence approximate membership, is achievable within a factor \(1 + \delta \) of \(n r\) bits for any fixed \(\delta > 0\), with efficient construction and constant-time queries; the Bloomier filter of Chazelle et al. [Cha+04] is the earlier retrieval-based design. Practical descendants include the xor filter [GL20], at about \(1.23\, n r\) bits, the binary fuse filter [GL22], closer still, and ribbon [Die+26]. Their false-positive rates need the same care as any filter’s: homogeneous ribbon, for instance, has exact rate \(p + (1 - p) 2^{-r}\), where \(p\) is the probability of a row-span event, so it is not the ideal \(2^{-r}\) of independent fingerprints. These, not the perfect hash filter, are the practical answer to the floor; the integer rounding of \(r\) remains unless fingerprints are coded jointly.

Comparison table

Table 14.1 collects the budgets. The perfect-hash column uses the idealized addressing cost \(\log _2 e\); a real construction adds between a few tenths of a bit and two bits more.

The BHF matches the floor but cannot be built at these sizes; the perfect hash filter is built in linear time and sits a constant above the floor; the Bloom filter is the simplest, and its gap to the floor grows with \(\log _2(1/\fprate )\). At the idealized addressing cost the Bloom filter is the largest in every row; with the notebook’s measured BBHash cost of \(3.23\) bits per key the perfect hash filter needs \(10.23\) bits per element at \(\fprate = 10^{-2}\) and is the larger of the two there. At \(n = 10^6\) and \(\fprate = 10^{-6}\) the perfect hash filter at its idealized cost saves about \(7.3\) bits per element over the Bloom filter, about \(0.9\) megabytes; with BBHash addressing the saving is about \(5.5\) bits per element.

14.4 False-Positive Rate Analysis

Both constructions have \(\fnrate = 0\) by construction, so their accuracy is their false-positive rate. For the BHF the rate is exact and the independence across non-members is inherited from the hash; the subtleties are what the construction may look at, and what happens when the acceptance set is chosen from the data. For the perfect hash filter the argument is the classical fingerprint-collision bound.

The BHF

The proposition places the BHF inside the model of Chapter 4: the non-member reports are independent Bernoulli\((\fprate )\) outcomes, as Axiom  1 requires, and the rate is a fixed parameter rather than a property of the particular salt. Two hypotheses carry the weight. The query must be fresh, not one of the points the build inspected; a construction that rejected salts according to how they treated some designated non-members would condition those non-members’ outcomes. And the acceptance set must be fixed before the salts are examined. The second hypothesis is the one the source manuscript’s adaptive variant gives up.

The perfect hash filter

For the perfect hash filter of Definition 14.3.1, let \(x \notin A\) and let the probability be over the fingerprint hash \(g\), with \(\phi \) fixed by the build. Two cases arise.

(a) \(\phi (x) = \phi (a^{*})\) for some \(a^{*} \in A\). This is the only case for a minimal perfect hash, whose range is exactly \([n]\). The query returns \(\top \) iff \(g(x) = g(a^{*})\), and since \(x \neq a^{*}\), uniformity of \(g\) on pairs gives probability \(2^{-r}\).

(b) \(\phi (x)\) falls outside \(\phi (A)\). This can happen only for a non-minimal perfect hash with spare slots. A slot holding a sentinel that matches no fingerprint contributes nothing; a slot holding an arbitrary value independent of \(g(x)\) contributes at most \(2^{-r}\).

The ceiling in \(r\) over-achieves the target by up to a factor of two: at \(\fprate = 0.01\), \(r = 7\) and the realized rate is \(2^{-7} \approx 0.0078\), about \(22\%\) below target, the same rounding that costs \(0.36\) bits per element on the space side. The BHF has no such rounding, because its acceptance set has density exactly \((t + 1)/N\).

The uniformity hypothesis

Both propositions rest on hash uniformity, and both fail against an adversary who knows the hash: a query chosen so that \(h(x, b^{*}) \le t\), or so that \(g(x) = g(a^{*})\), is a false positive with probability one. For the perfect hash filter the requirement is only pairwise collision probability, which the universal hashing of Carter and Wegman [CW79] supplies: drawing \(g\) from a universal family at build time bounds every pairwise collision probability by the reciprocal of the codomain size. The BHF’s analysis uses the random-oracle model, a stronger idealization, because the build examines exponentially many salts and the argument needs the hash values at all of them to behave independently; a real hash reduced modulo a modulus that is not a power of two is also only approximately uniform without rejection sampling. Salting with a build-time secret denies an adversary advance knowledge of the hash. Chapter 16 returns to adaptive and adversarial queries, where these averages stop describing the system.

14.5 Construction Time and Trade-offs

The two constructions of this chapter differ less in what they store than in how they find it. The BHF searches; the perfect hash filter addresses. This section prices the search, prices the addressing, and offers a decision rule that includes the Bloom filter and the retrieval filters of Section 14.3.

The BHF build is exponential

The build of Algorithm 6 examines an expected \(1/s = \fprate ^{-n}\) salts ((14.2)), each rejected after an expected at most \(1/(1 - \fprate )\) hash evaluations, so its expected work is about \(\fprate ^{-n}/(1 - \fprate )\) hash evaluations. In bits, the search costs \(2^{\ell ^{*}}\) trials for an \(\ell ^{*}\)-bit representation: the space optimality of Theorem 14.3.1 and the exponential build are the same fact read in two directions. At \(\fprate = 1/2\) a set of \(64\) members needs about \(1.8 \times 10^{19}\) salts; at \(\fprate = 0.01\) a set of \(10\) members already needs \(10^{20}\). The BHF is a construction for small sets.

That is how seed search is used in practice. Split \(A\) by a first-level hash into buckets of expected size \(\beta \) and search one salt per bucket. The work becomes about \((n/\beta ) \fprate ^{-\beta }\), and the space becomes the sum of the per-bucket salt lengths, \(n \log _2(1/\fprate )\) in expectation, plus the constant slack of Theorem 14.3.1 for each bucket and the cost of recording where each variable-length salt begins. The per-element overhead therefore falls like \(1/\beta \) while the work grows like \(\fprate ^{-\beta }\), and how cheaply the salts can be encoded jointly is a research question in its own right. Minimal perfect hashing uses exactly this design, searching small buckets for seeds that are injective rather than accepting [EGV20; LSW24], and Lehmann et al. [Leh+25] combine the search with the encoding of the seeds to reduce that overhead. The book does not develop a bucketed BHF; it uses the single-salt BHF as the construction that shows what the floor costs.

The perfect hash filter build is linear

The perfect hash filter replaces the search over all of \(A\) with an expected-linear minimal perfect hash construction followed by one pass writing fingerprints. Linear-time perfect hashing goes back to Fredman, Komlós, and Szemerédi [FKS84], whose scheme stores \(O(n)\) words; space-efficient minimal perfect hash functions with expected linear construction include the hypergraph-peeling construction of Botelho, Pagh, and Ziviani [BPZ07], hash-displace-and-compress [BBD09], and BBHash [Lim+17], at roughly \(2\) to \(3\) bits per key; RecSplit and ShockHash approach \(\log _2 e\) by spending more build time on bucketed seed search. The fingerprint pass evaluates \(g\) once per member and writes distinct slots, so it parallelizes trivially. Building costs a few hash evaluations per member with a constant expected number of retries, against the Bloom filter’s \(k\) hash evaluations and bit writes per member with none.

Query time

The BHF’s query is one hash evaluation and one comparison. The perfect hash filter’s query evaluates the perfect hash (a constant number of hashes and memory probes, the constant depending on the construction), one fingerprint hash, one table read, and one \(r\)-bit comparison. The Bloom filter’s query costs \(k\) hash evaluations and \(k\) reads, with \(k\) at the optimum equal to \[ k^{*} \;=\; (m/n) \ln 2 \;=\; \log _2(1/\fprate ), \] since \(m/n = \log _2(1/\fprate ) \cdot \log _2 e\) (Proposition 8.4.1) and \(\log _2 e \cdot \ln 2 = 1\); at \(\fprate = 0.01\), \(k^{*} \approx 6.6\), and at \(\fprate = 10^{-6}\), \(k^{*} \approx 19.9\). Both static structures answer in a number of memory probes that does not grow with \(\log _2(1/\fprate )\), which is often a more consequential advantage than the bits.

Trade-offs

All three structures are FNR-zero. The BHF and the perfect hash filter are static: inserting or deleting a member means rebuilding, while the Bloom filter inserts in \(O(k)\) and supports no deletion (clearing shared bits would create false negatives). Table 14.2 summarizes.

Decision rule

Three questions settle the choice: is the set static, is space binding, and how tight is the target?

Static set, space binding. Use a retrieval-based filter (xor, binary fuse, ribbon) or a perfect hash filter. The perfect hash filter beats the Bloom filter only when \((\log _2 e - 1) \log _2(1/\fprate )\) exceeds its addressing cost, about \(\fprate < 2^{-7}\) with BBHash addressing (Proposition 14.3.1); the retrieval filters approach the floor more closely at every target.

Dynamic set. Use a Bloom filter, or a cuckoo or quotient filter [Fan+14; Ben+12], which store fingerprints with an additive overhead of a few bits per element and support insertion and deletion. Pagh, Pagh, and Rao give a dynamic structure within a \(1 + o(1)\) factor of the floor plus \(O(n)\) bits [PPR05].

Large static base, small dynamic delta. Keep a static filter over the base and a small Bloom filter over the delta, report \(\top \) if either does, and rebuild the static part when the delta grows past a fixed fraction of the base.

The BHF itself is not on the list. It is the construction that attains the floor exactly and the reference against which the others are measured; at deployment scale it appears only as the per-bucket primitive of seed-search constructions.

14.6 Bridge

The chapter answered the optimality question of chapter 8 twice. The Bernoulli hash function (Definition 14.2.1) attains the entropy floor of Theorem 8.3.1 to leading order, with no integer rounding (Theorem 14.3.1) and an exact false-positive rate (Proposition 14.4.1), and pays for it with a build exponential in the set size. The perfect hash filter (Definition 14.3.1) builds in expected linear time and pays an additive addressing cost of at least \(\log _2 e\) bits per element (Proposition 14.3.1), which turns the Bloom filter’s multiplicative overhead (Proposition 8.4.1) into an additive one and wins on space at tight targets; retrieval-based filters close most of the remaining gap. None of these constructions originates here, and Section 14.7 names their sources.

All of them sit on the FNR-zero arm of the perfect-filter line (Definition 6.4.1): they never miss a member and occasionally accept a stranger. The other arm, \(\fprate = 0\) with a controlled \(\fnrate \), is the subject of chapter 15, and the two arms are not symmetric in cost. A false positive is a coincidence, a non-member’s hash falling into the acceptance set or its fingerprint matching the one stored in its slot, and about \(\log _2(1/\fprate )\) bits per element make the coincidence as rare as desired. A guarantee of no false positives is not a coincidence made rare but a certificate enforced against every non-member at once, and certifying a designated subset exactly is the expensive direction: over the whole universe it costs on the order of \(\log _2(|\Universe |/n)\) bits per element, the price of a dictionary. Applications split along the same line. FNR-zero structures serve where a miss is unacceptable, a cache that must not lose a stored key or a routing layer that must not drop a known address; FPR-zero structures serve where a spurious acceptance is unacceptable, a precision-oriented index or an authorization filter.

Chapter 15 reaches the FPR-zero arm by reusing the perfect hash filter’s skeleton, a minimal perfect hash addressing a per-slot table built in one pass, with a rank in each slot instead of a fingerprint. It is not a mechanical swap of error modes. Queried over the whole universe, the rank table behaves exactly like a fingerprint table, with no false negatives and false positives at \(2^{-r}\); the false-positive rate becomes zero only in a restricted query model in which non-members cannot present a valid rank (ranked search is the standard instance), and a controlled false-negative rate is then bought by relaxing the perfect hash to a \(k\)-perfect one. Chapter 15 develops the construction and its application.

14.7 Notes and Lineage

BF Lineage. The floor that frames the chapter, \(n \log _2(1/\fprate )\) bits for an FNR-zero approximate set, is due to Carter et al. [Car+78]; its two-sided extension, with a bounded fraction of false negatives, is settled to leading order by Pagh and Rodler [PR01, Prop. 1]. The BHF of Definition 14.2.1 follows the program’s manuscript [Tow26j], but its mechanism, one seed required to satisfy \(n\) independent Bernoulli constraints, with a seed index of \(\log _2(1/s) + O(1)\) bits and \(1/s\) expected trials, is the standard single-seed baseline of seed search. It is analyzed as such by Lehmann et al. [Leh+25, Sec. 1.1] and used on small buckets by RecSplit and ShockHash [EGV20; LSW24]. The perfect hash filter, a minimal perfect hash with \(\lceil \log _2(1/\fprate ) \rceil \)-bit fingerprints and its \(\log _2 e\) overhead, is described by Dietzfelbinger and Pagh [DP08, Sec. 1]; the \(n \log _2 e\) lower bound for minimal perfect hashing is Fredman and Komlós [FK84], and the constructions cited in Section 14.5 run from Fredman, Komlós, and Szemerédi [FKS84] through Botelho, Pagh, and Ziviani [BPZ07], Belazzougui, Botelho, and Dietzfelbinger [BBD09], and Limasset et al. [Lim+17]. Retrieval-based filters that approach the floor at linear build time are the Bloomier filter [Cha+04], the retrieval structures of Dietzfelbinger and Pagh [DP08], xor and binary fuse filters [GL20; GL22], and ribbon [Die+26]; dynamic alternatives are the cuckoo filter [Fan+14], the quotient filter [Ben+12], and the structure of Pagh, Pagh, and Rao [PPR05]. Fingerprint uniformity rests on the universal hashing of Carter and Wegman [CW79].

BF What this chapter adds. A presentation of the salt-search BHF and the perfect hash filter side by side as the two ends of a space-versus-build trade on the FNR-zero arm, with the BHF’s expected size bounded on both sides (Theorem 14.3.1) and the perfect hash filter’s break-even with the Bloom filter stated with real addressing costs. It also records two corrections to the source manuscript and to this book’s earlier drafts: the adaptive threshold trades recall rather than saving space, and its exact discrete law is a binomial tail (Remark 14.4.1); and the perfect-hash-addressed table, which an earlier draft of this chapter presented as the BHF, does not meet the floor, because no minimal perfect hash is sublinear.

BF Left out. Bucketed seed-search encodings and their analysis, dynamic and expandable filters, cache-efficient layouts, and measured throughput. The map extension appears only through its success probability; the source manuscript develops it.

Chapter 15
Perfect Hash Filters (BPHF)

15.1 From the Perfect Hash Filter to the BPHF

Chapter 14 settled the FNR-zero arm of the perfect-filter line. This chapter turns to the FPR-zero arm. The same architectural template, a perfect hash addressing a per-element table, produces a structure with the dual error behavior: every report of membership is correct, and the only failure mode is silent omission. The two chapters together give the constructive realization of both arms of Definition 6.4.1, and neither arm is rhetorical or structurally easier than the other, as the asymmetry this chapter develops will show.

Recall the FNR-zero constructions

Chapter 14 gave two constructions on the FNR-zero arm. The Bernoulli hash function of Definition 14.2.1 stores a single salt under which every member falls in a fixed acceptance set; it attains the entropy floor of Theorem 8.3.1 to leading order (Theorem 14.3.1) but builds by a search exponential in the set size. The perfect hash filter of Definition 14.3.1 addresses a fingerprint table \(F : [n] \to \{0, 1\}^r\) through a minimal perfect hash \(\phi : A \to [n]\) and compares the stored fingerprint \(F[\phi (x)]\) with \(g(x)\); at \(r = \lceil -\log _2 \fprate \rceil \) it has \(\fprate \le 2^{-r}\) (Proposition 14.4.2) at \(r\) bits per element for the table plus at least \(\log _2 e\) for the perfect hash (Proposition 14.3.1). That second construction, standard since at least Dietzfelbinger and Pagh [DP08, Sec. 1], is the skeleton of this chapter. Chapter 14 closed with one question deferred: what happens on the other axis?

The opposite axis

The FNR-zero arm is the home for applications that cannot tolerate a missed member: a routing table that must never drop a known destination, a security check that must never admit a denied identifier, a cache that must never miss a stored key. A second class inverts the trade-off. Top-\(k\) retrieval must never surface an off-topic document as if it belonged in the top set, but can tolerate occasionally missing one that should have been included. A precision-oriented search index must never claim a non-matching record as a match, though the system can recover from an undercount by relaxing the query. A deduplication layer must never declare two distinct items the same, but a few surviving duplicates are recoverable by a later pass. In each, a false positive corrupts the output irrecoverably and a false negative is a tolerable miss the surrounding system absorbs. The desired guarantee flips: \(\fprate = 0\) exactly, with \(\fnrate > 0\) the controllable cost. Proposition 6.4.2 already established that the FPR-zero arm is closed under union and intersection, mirroring the FNR-zero arm; the algebraic shape is symmetric. What the present chapter supplies is the constructive question that shape leaves open: which structure achieves \(\fprate = 0\) at a controllable \(\fnrate \), and at what space cost.

Construction sketch

The construction reuses the minimal perfect hash \(\phi : A \to [n]\) of the perfect hash filter unchanged and substitutes the second ingredient. In place of a fingerprint table storing \(g(a)\), the BPHF stores a rank value \(\rho (a)\) at slot \(\phi (a)\), an integer in \([0, n)\) encoding \(a\)’s position in some build-time ordering of \(A\), derived from a secondary hash or from external metadata such as a relevance score. The rank table \(R : [n] \to \{0, 1\}^r\) uses \(r = \lceil \log _2 n \rceil \) bits per slot, enough to distinguish the \(n\) ranks. A query computes \(\phi (x)\), reads \(R[\phi (x)]\), and returns \(\top \) when the stored value equals \(\rho (x)\).

The substitution looks like a free relabeling, and Section 15.2 shows it is not. Taken at face value over the whole universe, the rank comparison reproduces the perfect hash filter rather than flipping its axis: a non-member still collides with the stored rank with probability \(2^{-r}\). The \(\fprate = 0\) guarantee appears only in a restricted query model, where \(\rho \) is a verifiable witness that genuine members carry and non-members cannot present. Ranked search is the standard instance of that model, and a controlled \(\fnrate \) is then bought not by the rank comparison but by relaxing the perfect hash itself. The next section makes each claim precise and locates the asymmetry between the two arms of the line.

15.2 The BPHF Construction

Section 15.1 set the goal: a structure on the FPR-zero axis, mirroring the way the perfect hash filter sits on the FNR-zero axis. The plan keeps the minimal perfect hash \(\phi : A \to [n]\) and replaces the fingerprint table with a rank table the query verifies against the candidate’s own rank. The construction is short; the honest accounting of what it does and does not deliver is the heart of the chapter, because the rank table alone does not flip the error axis, and seeing why reveals the asymmetry between the two arms of the perfect-filter line.

The two ingredients

Fix a finite universe \(\Universe \) and a designated set \(A \subseteq \Universe \) with \(|A| = n\).

(i) The perfect hash \(\phi \). The same minimal perfect hash \[ \phi \;:\; A \;\longrightarrow \; [n], \qquad [n] = \{1, 2, \ldots , n\}, \] that the perfect hash filter of Definition 14.3.1 uses: an injection on \(A\), hence a bijection onto the addressing range, constructed at build time from \(A\) and extended to all of \(\Universe \). Its build and storage costs are the perfect hash filter’s (Section 15.3). The BPHF reuses it without modification.

(ii) The rank table \(R\) and assignment \(\rho \). A rank table \(R : [n] \to \{0, 1\}^r\) of \(n\) slots, each of width \begin {equation} \label {eq:bphf-r} r \;=\; \lceil \log _2 n \rceil , \end {equation} the smallest integer with \(2^r \ge n\). It is populated by a rank assignment \(\rho : A \to \{0, 1, \ldots , n-1\}\) giving each member a distinct integer, from a position in a sorted ordering, a relevance score discretized to \(n\) buckets, or external metadata. Distinctness of the ranks is what fixes \(r\) at \(\lceil \log _2 n \rceil \): it takes that many bits to label \(n\) elements apart.

Where the perfect hash filter compared a stored fingerprint \(g(a)\) against the candidate’s \(g(x)\), the BPHF compares a stored rank \(\rho (a)\) against the candidate’s \(\rho (x)\). The substitution is one line of pseudocode. Whether it changes the error analysis is the question the rest of the section answers, and the answer is more subtle than the symmetry of the picture suggests.

Construction and query

Given \(A\), \(\phi \), and \(\rho \), the table is built in one pass: \begin {equation} \label {eq:bphf-build} \text {for each } a \in A: \quad R[\phi (a)] \;\coloneqq \; \rho (a), \end {equation} writing each slot exactly once by bijectivity of \(\phi \), so the table is fully populated.

The definition has the shape of Definition 14.3.1: a perfect-hash addressing function, a per-slot value, and a query that reads one slot and compares it to a computed value. The only change is which value is stored, a uniform fingerprint \(g\) for the perfect hash filter, an application-chosen rank \(\rho \) for the BPHF. Algorithm 8 writes it out.

Against the perfect hash filter’s pseudocode (Algorithm 7), the only changed line substitutes \(\rho \) for \(g\) in the value compared. The two algorithms share their performance profile. Whether they share their error profile is the question, and it needs care.

What the bare construction delivers

Read the algorithm directly. For a member \(a \in A\), bijectivity sends \(a\) to a slot written by \(a\) alone, where the build stored \(\rho (a)\); the member recomputes \(\rho (a)\), reads its own slot, and matches. The match always succeeds, so \(\fnrate = 0\). Even two members with equal ranks would not interfere, because \(\phi \) sends them to different slots and each reads back the rank it wrote.

For a non-member \(x \notin A\), the extended hash sends \(x\) to a slot \(\phi (x) = \phi (a^{*})\) owned by some member \(a^{*}\), holding \(\rho (a^{*})\). The query accepts \(x\) exactly when the recomputed \(\rho (x)\) equals \(\rho (a^{*})\). If \(\rho \) behaves like a uniform \(r\)-bit hash, the two agree with probability \(2^{-r}\), so \(\fprate = 2^{-r}\), exactly the fingerprint-collision probability of Proposition 14.4.2. The bare BPHF, queried over the full universe with a recomputable rank, is the perfect hash filter of Definition 14.3.1 at width \(r = \lceil \log _2 n \rceil \): zero false negatives, false positives at \(2^{-r}\). Renaming a fingerprint a rank changes nothing the query can observe. Figure 15.1 shows the shared layout.

The perfect hash filter is not this chapter’s invention. The pairing of a minimal perfect hash with \(\lceil \log _2(1/\fprate ) \rceil \)-bit fingerprints is the standard static approximate-membership construction, described with its \(\log _2 e\) overhead by Dietzfelbinger and Pagh [DP08, Sec. 1], who note that its origin is unknown to them; the program’s manuscript [Tow26m] restates it as a Bernoulli set and lifts it to maps. It is the practical FNR-zero construction, as against the salt-search BHF of Definition 14.2.1: the BHF attains the floor \(-\log _2 \fprate \) to leading order but builds by exponential search, while the perfect hash filter pays at least \(\log _2 e \approx 1.44\) bits per element above the floor (Proposition 14.3.1) and builds in expected linear time by off-the-shelf minimal perfect hashing. Because the overhead is additive, it is a vanishing fraction of the cost as \(\fprate \to 0\); at loose targets it is not, and a Bloom filter can be smaller (Section 14.3). The rank table of Definition 15.2.1 is this same construction read on the FPR-zero axis: it fixes \(r = \lceil \log _2 n \rceil \) so that each slot can carry a distinct rank, and the witness model of the next subsection turns the result into an exact filter.

This is the asymmetry Section 15.1 promised. A false positive is a coincidence, one stored value colliding with one recomputed value, and a single \(r\)-bit slot makes it rare; a false-negative-free filter is therefore cheap, bought for about \(\log _2(1/\fprate )\) bits per element plus the addressing. The FPR-zero axis asks the opposite, that no non-member is ever accepted, and that is not a coincidence to make rare but a certificate to enforce against every one of the \(|\Universe | - n\) non-members at once. Certifying a subset exactly is the expensive direction. The next two subsections give the two honest routes to it.

How the false-positive rate becomes zero

The bare construction admits a non-member only when its recomputed rank equals the rank stored at the slot it lands in. There is exactly one way to close that gap without storing a universe’s worth of information: arrange the query so a non-member never presents a matching rank in the first place. Call \(\rho \) a witness for \(A\) when the recomputed \(\rho (x)\) equals the stored rank at slot \(\phi (x)\) if and only if \(x\) is the member that claimed that slot. Under a witness, \(R[\phi (x)] = \rho (x)\) holds only for members, so \[ \fprate \;=\; 0 \] holds exactly, by construction rather than in expectation. The relevance ranking of a search corpus is the standard example, worked in Section 15.4: a query is posed only for a document that already carries its corpus rank, an outside document has no such rank to present, and the witness condition is met.

The witness model is not a free upgrade. It restricts the query domain to candidates that arrive with a verifiable rank, and it pays for the guarantee in the only currency that buys it: the rank table stores, in \(n\) slots of \(\lceil \log _2 n \rceil \) bits, enough to name every member apart and certify a presented witness against the right slot. That is the cost of certifying a subset exactly, the same counting argument that set the book’s earlier space bounds. A structure that accepts no non-member must in effect encode which elements it accepts, and encoding a designated subset of an arbitrary universe costs on the order of \(\log _2 n\) bits per element once the query domain is the corpus itself. A genuinely universe-wide FPR-zero filter, rejecting every outsider by recomputation alone, must distinguish \(A\) among all \(\binom {|\Universe |}{n}\) candidate sets and so stores about \(\log _2(|\Universe |/n)\) bits per element; at that price it is a dictionary, not a compact filter.

The witness model is a property of the query, not of the table, and the distinction has teeth.

With distinct ranks and a true perfect hash, the witness model places the BPHF at the corner of the perfect-filter line: both error rates are zero on the query domain. The structure is exact there, and exactly for that reason not yet an approximate set. The mirror of the perfect hash filter, a tunable structure trading a controlled error for space, comes from relaxing the perfect hash.

A controlled false-negative rate via k-perfect hashing

The perfect hash filter moves along its axis by changing the fingerprint width. The BPHF needs the matching dial on its own axis, a way to spend less space for a controlled, nonzero \(\fnrate \) while holding \(\fprate = 0\). The dial is the perfect hash. A \(k\)-perfect hash relaxes injectivity, allowing up to \(k\) members to share a slot; such hashes are standard [BBD09]. A \(k\)-perfect hash into \(m < n\) slots is cheaper to store, but a shared slot holds only one member’s rank; the members that lose the contention read a different member’s rank, fail the comparison, and are reported as non-members. These displaced members are the only error the \(k\)-perfect BPHF makes, and a non-member still cannot present a witness, so \(\fprate \) stays exactly zero. If the construction occupies \(o \le m\) slots, exactly \(n - o\) members are displaced, so \begin {equation} \label {eq:bphf-fnr} \fnrate \;=\; 1 - \frac {o}{n} \;\ge \; 1 - \frac {m}{n}, \end {equation} with equality when every slot is occupied, zero at \(m = n\) (the exact corner), and growing as the table shrinks. A table at ninety percent of full size, fully occupied, misses one member in ten while admitting no non-member at all.

Three cautions keep the dial honest. First, the misses are not independent across members: they are decided slot by slot, exactly one of the members sharing a slot surviving, so the member side does not satisfy element-wise independence (Axiom 1). The fraction \(1 - o/n\) is a property of the build, not a per-element coin. Second, a formula for the expected number of displaced members under random placement must use the same conditioning as the construction: a build that rejects placements with an overfull slot has, for example, no displaced members at all when \(m = n = 2\) and \(k = 1\), whereas the unconditional occupancy law gives an expected displaced fraction of \(1/4\). Third, in the fingerprint reading of the same skeleton a displaced member is rescued whenever its fingerprint happens to equal the slot owner’s, so its false-negative rate is the displaced fraction times \(1 - 2^{-r}\); with one-bit fingerprints and \(k = 2\), a displacement probability of \(1/4\) gives a membership false-negative rate of \(1/8\). Dropping members outright, as in the lossy dictionaries of Pagh and Rodler [PR01], achieves a given \(\fnrate \) in fewer bits per retained member; the \(k\)-perfect variant buys construction simplicity rather than space.

This is the mirror of the perfect hash filter made precise: the perfect hash filter holds \(\fnrate = 0\) and dials \(\fprate \) down by adding fingerprint bits; the BPHF holds \(\fprate = 0\) in the witness model and dials \(\fnrate \) down by adding slots. Each lives on its own axis of Definition 6.4.1, paying for accuracy in space by Proposition 6.4.2.

A worked example

Fix \(n = 1000\). The rank width is \[ r \;=\; \lceil \log _2 1000 \rceil \;=\; \lceil 9.966 \rceil \;=\; 10 \text { bits}, \] since \(2^{10} = 1024 \ge 1000 > 2^9 = 512\). The rank table occupies \(n r = 10{,}000\) bits (\(1250\) bytes), plus a minimal perfect hash of at least \(\log _2 e \approx 1.44\) and in practice \(2\) to \(3.3\) bits per element (Proposition 14.3.1), for a total of at least about \(11{,}440\) bits and in practice \(12{,}000\) to \(13{,}300\) bits. At the corner, a true perfect hash with distinct ranks in the witness model, \(\fprate = \fnrate = 0\): the structure never accepts a non-corpus candidate and never misses a corpus member, a perfect-hash dictionary viewed as the corner of the line. To make it a tunable approximate set, shrink to a \(k\)-perfect hash with \(m = 900\) fully occupied slots. By (15.3), \[ \fnrate \;=\; 1 - \tfrac {900}{1000} \;=\; 0.10, \qquad \fprate \;=\; 0, \] so one corpus member in ten is missed while no outside candidate is ever accepted. For comparison, a perfect hash filter on the same set at \(\fprate = 0.01\) uses \(r = 7\) bits per element for its table (\(7000\) bits, plus the same perfect hash) and gives \(\fnrate = 0\) at \(\fprate \le 0.0078\). The two answer opposite needs: the perfect hash filter never misses a member and rarely admits a stranger; the BPHF never admits a stranger and occasionally misses a member. An application that can tolerate neither must pay the dictionary price for both. Section 15.4 works through one setting where the BPHF’s profile is exactly the one the problem wants.

15.3 Space and Time Analysis

The BPHF stores the minimal perfect hash \(\phi \) and the rank table \(R\) of width \(r = \lceil \log _2 n \rceil \). This section totals the bits, gives the build and query times the construction inherits from the perfect hash filter, and tabulates the trade against it. The headline is Proposition 15.3.1: the per-element cost is \(\lceil \log _2 n \rceil \) for the ranks plus the perfect hash’s addressing cost, which is at least \(\log _2 e\). The build is expected \(O(n)\) and queries \(O(1)\), as for the perfect hash filter. The structural difference is on the space axis: the BPHF’s bits per element grow with the set size \(n\), the perfect hash filter’s with the target precision \(\log _2(1/\fprate )\), so the two are not substitutes parameter for parameter.

Rank table storage

The table \(R : [n] \to \{0, 1\}^r\) holds \(n\) slots of \(r\) bits, so \begin {equation} \label {eq:bphf-rank-bits} L_R \;=\; n \cdot r \;=\; n \cdot \lceil \log _2 n \rceil \end {equation} bits, growing as \(n \log _2 n\), the cost any index pays to distinguish \(n\) ranks. At \(n = 10^6\) the table is \(20 \cdot 10^6\) bits, about \(2.5\) megabytes; at \(n = 10^9\) it is \(30 \cdot 10^9\) bits, about \(3.75\) gigabytes. The integer rounding in \(r\) adds between \(0\) and \(1\) bits per element over \(\log _2 n\), as it does for fingerprints in the perfect hash filter.

Perfect-hash storage

The perfect hash depends on the set, not on the values stored at each slot, so the BPHF inherits the perfect hash filter’s addressing cost unchanged: \(L_\phi \ge n \log _2 e - O(\log n)\) bits for any minimal perfect hash function [FK84], and in practice from about \(1.5\) bits per key (RecSplit, ShockHash) to about \(3\) (BBHash), with hash-displace-and-compress near \(2\) (Proposition 14.3.1). The rank pass populates \(R[\phi (a)] = \rho (a)\) where the perfect hash filter populated \(F[\phi (a)] = g(a)\); the representation, slot count, and addressing are identical.

Total bits per element

Combining, \begin {equation} \label {eq:bphf-total-bits} L_{\text {BPHF}} \;=\; L_R + L_\phi \;=\; n \cdot \lceil \log _2 n \rceil + L_\phi , \qquad L_\phi \;\ge \; n \log _2 e - O(\log n), \end {equation} and dividing by \(n\), with \(c_\phi = L_\phi / n\) the addressing cost per element, \begin {equation} \label {eq:bphf-per-element} L_{\text {BPHF}} / n \;=\; \lceil \log _2 n \rceil + c_\phi , \qquad c_\phi \;\ge \; \log _2 e - o(1). \end {equation}

The proposition has the form of Proposition 14.3.1 with one difference: the leading term is controlled by the set size \(n\), not the target rate \(\fprate \). The two budgets \[ L_{\text {PHF}} / n = \lceil -\log _2 \fprate \rceil + c_\phi , \qquad L_{\text {BPHF}} / n = \lceil \log _2 n \rceil + c_\phi \] share the addressing term and differ in the slot width. The perfect hash filter spends slot bits at a rate the designer sets through \(\fprate \); the BPHF at a rate the set size dictates. The perfect hash filter pays \(\lceil -\log _2 \fprate \rceil \) bits per slot to make a false-positive coincidence rare; the BPHF pays \(\lceil \log _2 n \rceil \) bits per slot so the rank assignment can name all \(n\) members apart and certify a witness. At full resolution the BPHF is exact (the \(\fnrate = 0\) corner); the \(k\)-perfect dial of Section 15.2 shrinks the slot count to \(m < n\) and trades \(\fnrate \ge 1 - m/n\) for the saved space.

Build and query time

The build writes one rank per element in a one-pass \(O(n)\) stage plus the expected-\(O(n)\) minimal perfect hash construction of Section 14.5, dominated by the latter; build throughputs match the perfect hash filter’s. The rank fill is embarrassingly parallel (distinct slots by bijectivity), running after the perfect-hash stage. The only build-time difference from the perfect hash filter is computing a rank \(\rho (a)\) rather than a fingerprint \(g(a)\) per element, both \(O(1)\), though computing the ranks themselves may require a sort.

The query of Algorithm 8 runs in \(O(1)\): one perfect-hash evaluation, one rank evaluation, one memory read, and an \(r\)-bit comparison (at machine-word speed for any \(n\) up to \(2^{64}\)). The cycle count is essentially the perfect hash filter’s, differing only in which function is the second evaluation and what the table read returns.

The bit-cost picture at \(n = 10^6\)

Fix \(n = 10^6\). The BPHF’s slot width is \(\lceil \log _2 10^6 \rceil = 20\) bits, independent of any error target; the perfect hash filter’s is \(\lceil -\log _2 \fprate \rceil \). Table 15.1 puts them side by side; both add the same addressing cost \(c_\phi \).

The two filters cross over at \(\fprate = 1/n\), where \(\lceil -\log _2 \fprate \rceil = \lceil \log _2 n \rceil \): at \(n = 10^6\) this is \(\fprate = 2^{-20} \approx 10^{-6}\). Above the crossover (looser targets, \(\fprate > 1/n\)) the perfect hash filter is cheaper; below it (stricter targets) the BPHF is cheaper at the same \(n\). The two regimes are the two operating modes, the perfect hash filter where the application tolerates some false positives for cheap storage, the BPHF where it cannot tolerate any and the witness model holds. The BPHF’s extra bits in the loose-target regime are not wasted: they buy a guarantee that no fingerprint width delivers. A perfect hash filter at \(\fprate = 10^{-2}\) admits about one false positive per hundred non-member queries; a BPHF at the same \(n\) admits none from candidates without a valid rank. The \(13\) extra bits per element are the price of converting a \(10^{-2}\) probability into a certainty on the consequential axis, the conversion Section 15.4 shows paying for itself in top-\(k\) retrieval.

15.4 Ranked Search Application

The chapter developed the BPHF abstractly: zero false positives in the witness model, exact at full resolution, a controlled \(\fnrate \ge 1 - m/n\) bought by a \(k\)-perfect hash of \(m\) slots, at \(\lceil \log _2 n \rceil \) bits per element plus the addressing cost (Proposition 15.3.1). This section places it in its archetypal application, which also supplies the witness model the guarantee needs. The setting is ranked search: given a corpus and a query, return the top-\(k\) most relevant documents. The \(\fprate = 0\) guarantee is exactly the precision invariant the application needs, and the BPHF is not a generic structure that happens to fit ranked search but the structure ranked search asks for.

The ranked-search problem

Fix a corpus of \(n\) documents with IDs \(a_1, \ldots , a_n\) and a relevance score \(s : \text {ID} \to \mathbb {R}\) from term weights, embeddings, or metadata. A query asks for the top-\(k\) IDs by score. The exact solution scores every document and sorts, at \(O(n)\) score evaluations plus \(O(n \log n)\) sorting, dominated by the scoring when \(s\) is expensive. At corpus sizes in the millions the question becomes whether top-\(k\) retrieval can avoid scoring every document at every query. A structure that pre-computes the answer once and serves queries without per-document scoring is the architectural answer, and the BPHF is one.

BPHF for top-\(k\) retrieval

The build is a single pass over the corpus: pre-compute \(s(a_i)\) for every document, sort the IDs by descending score, and assign each its rank, the top-scoring ID rank \(0\) and the bottom rank \(n - 1\). That rank assignment \(\rho : A \to \{0, \ldots , n-1\}\) is exactly the input Section 15.2 requires; build the BPHF on it with \(r = \lceil \log _2 n \rceil \) bits per slot. Top-\(k\) retrieval reduces to a rank-bound check: a candidate \(a\) is in the top \(k\) iff \(\rho (a) < k\), so \[ \textsc {TopK}(a) \;=\; \textsc {Query}(a) \;\land \; R[\phi (a)] < k, \] combining the BPHF membership check with a numeric comparison. The query is \(O(1)\): one perfect-hash evaluation, one rank evaluation, one memory read, one \(r\)-bit comparison, one integer comparison against \(k\), constant in both \(n\) and \(k\). The build is the one-time \(O(n \log n)\) sort plus the \(O(n)\) BPHF build of Section 15.3; each query then saves the \(n\) score evaluations the exact solution repeats, dropping per-query cost from \(O(n)\) to \(O(1)\). The profile is batch build and constant-time query, matching the perfect hash filter’s deployment: a search engine rebuilds on each hourly corpus snapshot and serves millions of queries per second between rebuilds; a recommender rebuilds per-user indices nightly and serves personalized top-\(k\) instantly.

Why \(\fprate = 0\) is the right guarantee

Top-\(k\) retrieval has an asymmetric error structure. A false positive surfaces an off-topic document as if it belonged in the top set: the user sees an obvious irrelevance, clicks it, finds it useless, and judges the system broken. The cost is corrupted output, visible and attributable. A false negative omits a document that should have appeared: the user sees a top-\(100\) list that is really a top-\(99\)-plus-one-missing, usually does not notice, and recovers the entry by relaxing the query when they do. The cost is mild under-recall, mostly invisible and recoverable. The asymmetry tracks the BPHF exactly: the \(\fprate = 0\) guarantee of Definition 15.2.1 eliminates the high-cost error by construction, and the \(\fnrate > 0\) trade admits the low-cost error at a controlled rate. The designer who chooses the BPHF makes the trade explicitly, zero corrupted output for a small under-recall, the right trade wherever precision is the binding metric and recall the recoverable one: ranked search, recommendation, precision-sensitive classification, authorization filters.