Lecture 17
Token sets · diffusion noise · singular supports · search agents
Vishnu Boddeti
Connect to the lecture notes
Use the notes’ set-based definition: $\nu(A)=\int_A f\,d\mu$. Every calculation below concerns a set and its mass.
Example 1
A finite set lets us see exactly how a measure changes when some outcomes are removed.
Example 1 · What is the mathematical object?
Use a four-token vocabulary $\Omega$. The measurable sets are all subsets of $\Omega$.
The original measure assigns masses $0.45,\ 0.30,\ 0.15,\ 0.10$ to the four singletons.
$P(A)=\sum_{a\in A}P(\{a\})$
Sort tokens by mass. Keep the shortest prefix whose combined mass reaches the cutoff.
Divide each retained mass by $P(N)$; assign zero to removed tokens.
$Q(\{a\})=\dfrac{P(\{a\})}{P(N)}\,\mathbf1_N(a)$
Before moving the control: can removing tokens create mass on a source-null set?
Example 1 · An exact four-token calculation
| Token | $P$ | Nucleus? | $Q$ | $Q/P$ |
|---|---|---|---|---|
| The | 0.45 | |||
| A | 0.30 | |||
| This | 0.15 | |||
| Other | 0.10 |
At cutoff 0.75, the first two tokens carry mass 0.75. Their new masses are 0.60 and 0.40.
Try cutoff 1: all tokens return and every ratio is 1.
Whenever tokens are removed, $Q\ll P$, but $P\ll Q$ fails. Which singleton witnesses the failure?
Example 2
The density is a function; the measure assigns mass to a set by integrating that function.
Example 2 · A one-coordinate view of the forward corruption
The noisy coordinate has a Gaussian density. The control changes its scale $\sigma$.
$f_\sigma(x)=\dfrac{1}{\sigma\sqrt{2\pi}}e^{-x^2/(2\sigma^2)}$
The demo isolates this kernel; it does not simulate an entire image or a trained denoising model.
Let $\lambda$ denote Lebesgue measure on the line.
$Q(A)=\int_A f_\sigma(x)\,d\lambda(x)$
For $A=[-0.5,0.5]$, mass is shaded area. A singleton has zero length, so $Q(\{0\})=0$ even when $f_\sigma(0)>1$.
Predict: does a taller, narrower peak put more or less mass inside this fixed interval?
Example 2 · Compute an interval’s mass
Shading marks $A=[-0.5,0.5]$. The vertical axis rescales to show the full peak; the displayed window is $[-3,3]$.
Try σ = 0.15. Density height can exceed 1; the integral over the whole line remains 1.
Example 3
Before writing a density ratio, check whether a set with zero source mass has positive target mass.
Example 3 · The paper’s parallel-line construction
$P_0$ spreads unit mass evenly along $\{0\}\times[0,1]$.
$P_\theta$ spreads unit mass evenly along $\{\theta\}\times[0,1]$.
Each segment has zero two-dimensional area. These measures therefore have no density relative to planar Lebesgue measure.
Let $A_\theta=\{\theta\}\times[0,1]$. If $\theta\ne0$,
$P_0(A_\theta)=0,\qquad P_\theta(A_\theta)=1.$
Thus $P_\theta\not\ll P_0$. No function integrated against $P_0$ can put mass on this source-null set.
If the shift becomes exactly zero, does a derivative relative to $P_0$ become possible?
Example 3 · Inspect the null-set condition
The segments’ drawn thickness is for visibility. Their two-dimensional area is zero.
Try θ = 0. The measures coincide, so $dP_\theta/dP_0=1$ almost everywhere—even though neither has an area density.
Example 4
A new action with positive mass cannot be represented by reweighting a measure that excluded that action.
Example 4 · Apply the Radon–Nikodym theorem from the notes
Use a small action set: Search A, Search B, Answer now, and Search C.
$P$ is the old action measure; $Q$ is the proposed one. An old logging filter excluded query C, so $P(\{C\})=0$.
The four actions and numeric masses are a teaching example, not a reproduction of Search-R1’s training procedure.
If $Q\ll P$, the theorem gives $w=dQ/dP$ with
$Q(A)=\int_A w\,dP.$
For this finite action set, $w(a)=Q(\{a\})/P(\{a\})$ where $P(\{a\})>0$.
On a source-null set, a finite density cannot create new mass.
Example 4 · Agentic support check
| Next action | Old $P$ | New $Q$ | $Q/P$ |
|---|---|---|---|
| Search A | 0.50 | ||
| Search B | 0.30 | ||
| Answer now | 0.20 | ||
| Search C | 0 |
Ratios on shared support reconstruct only the mass assigned to the first three actions. A positive query-C mass prevents a derivative for the whole target measure.
For $A=\{\text{Search B},\text{Answer now}\}$, verify $Q(A)=\sum_{a\in A}P(\{a\})w(a)$.
Return to the lecture’s central question
Before writing $dQ/dP$, name $P$, name $Q$, and test the null-set condition.
Each scenario hides a mathematical structure from this lecture. Identify the structure, justify it, and work through the resulting problem.
Every time an image is used for training, the pipeline chooses one of three crops and may flip it. A reproducibility audit asks a deceptively simple question: what, exactly, is the random experiment?
Use the six crop-flip pairs in $\{1,2,3\}\times\{0,1\}$ as equally weighted elementary outcomes: assign mass $1/6$ to each pair. The second entry records whether the image is flipped.
A few large residuals keep appearing, so a Gaussian model feels too unforgiving. You try a density proportional to $e^{-|r|}$ instead—but “proportional to” is not yet a probability model.
$p(r)=ce^{-|r|}$ for $r\in\mathbb{R}$.
Keep one search-agent prompt and tool history fixed. Four possible next actions represent three search queries and finishing. Compare a logging measure P with a proposed measure Q on this same finite action set.
$\Omega=\{A,B,C,F\}$, where $A,B,C$ are search queries and $F$ means finish. Assign $P=(0.5,0.2,0.2,0.1)$ and $Q=(0.3,0.3,0.3,0.1)$ in that order.
Take a unit circle and draw a chord uniformly at random. What is the probability that this “random chord” is longer than $\sqrt{3}$?