Lecture 18
Sparse connections · token sampling · SQL parsing · agent traces
Vishnu Boddeti
Specify $(\Omega,\mathcal A,P)$. An outcome might be a uniform draw or a complete agent trace.
A probability belongs to a measurable subset of $\Omega$.
A measurable map $X$ assigns a value to each outcome.
$P_X(A)=P(X^{-1}(A))$
The preimage identifies the original outcomes whose values lie in $A$.
$F_X(x)=P(\{\omega:X(\omega)\le x\})$
The CDF accumulates the mass of values at or below $x$.
Choose an event $B$ with $P(B)>0$, restrict to its outcomes, and renormalize.
$P(A\mid B)=\dfrac{P(A\cap B)}{P(B)}$
Mass $1/2$ at 0 and mass $1/2$ at 1.
Uniform mass on $[-1,1]$.
Mass $1/2$ at 0 and density $1/2$ on $(0,1)$.
The jump $F_X(x)-F_X(x^-)$ equals $P(X=x)$. A filled endpoint gives the CDF value.
A CDF determines a probability measure on $\mathbb R$. A density describes its absolutely continuous component.
Example 1
Endpoint masses and a continuous component make both exact sparsity and variable connection strengths possible.
Hard-concrete gates · The distribution induced by clipping
The paper learns the distribution of a continuous gate variable $S$. Clipping gives
$Z=\min(1,\max(0,S)).$
The whole lower tail maps to one point. This creates the singular, atomic component of the output measure.
$P(Z=0)=F_S(0)$
$P(0<Z<1)=F_S(1)-F_S(0)$
$P(Z=1)=1-F_S(1)$
The connection is active when $Z>0$. Its activity probability is $1-F_S(0)$, which enters the paper's sparsity penalty.
Hard-concrete gates · Calculate the three components
For $\gamma<t<\zeta$, $F_S(t)=\sigma\!\left(\beta\log\frac{t-\gamma}{\zeta-t}-a\right)$.
$\sigma(v)=1/(1+e^{-v})$, $\beta=2/3$, $\gamma=-0.1$, $\zeta=1.1$.
Try moving $a$ left and right. Identify the two jumps and check that the three masses sum to 1.
Example 2
The intervals mapped to each token make the random variable and its induced distribution visible.
Token decoding · Give the random variable a numeric value
| $i$ | Token | $p_i$ at $T=1$ | $F_X(i)$ |
|---|---|---|---|
| 1 | The | 0.5 | 0.5 |
| 2 | A | 0.3 | 0.8 |
| 3 | This | 0.2 | 1.0 |
These illustrative masses describe a small vocabulary at a fixed prompt. The indices provide an ordering for the CDF.
Use length measure on $\Omega=(0,1]$ and let $U$ be its coordinate.
$X(u)=\min\{i:F_X(i)\ge u\}.$
The preimage of token 2 is $(0.5,0.8]$. Its length is $0.3$, giving $P(X=2)=0.3$.
At $u=0.65$, the decoder emits token 2.
Where would $u=0.90$ land? Which set of uniform draws emits token 3?
Token decoding · Follow one uniform draw
| Token | $p_i$ | $F_X(i)$ | Preimage of $i$ |
|---|---|---|---|
| The | |||
| A | |||
| This |
Moving $U$ changes the selected outcome. Changing $T$ changes the interval lengths through $p_i\propto e^{\log(p_i^{\rm base})/T}$.
Example 3
Conditioning on the valid-token event changes the denominator used to normalize the candidate masses.
PICARD · Restrict to admissible outcomes
students(name, age)courses(title, credits)The prefix is the query text written so far. Our teaching grammar expects a column after SELECT and a table after FROM.
The four candidate masses are $0.4,0.3,0.2,0.1$ for name, age, students, courses.
After SELECT, $G=\{\text{name},\text{age}\}$, with $P(G)=0.7$.
$Q(A)=P(A\mid G)=\dfrac{P(A\cap G)}{P(G)}$
Thus $Q(\text{name})=4/7$ and $Q(\text{age})=3/7$.
The numbers and vocabulary are illustrative. The paper supplies the incremental parsing mechanism.
PICARD · Calculate the conditional distribution
SELECT ▌
| Candidate | Base $P$ | Valid? | $P(\cdot\mid G)$ |
|---|---|---|---|
| name | 0.40 | ||
| age | 0.30 | ||
| students | 0.20 | ||
| courses | 0.10 |
The parser chooses the admissible subset at the current prefix. Each surviving mass is divided by that subset's total mass.
Switch to FROM. Identify the retained rows, add their base masses, and check that the conditional column sums to 1.
students(name, age)courses(title, credits)Example 4
A structured trace can be an outcome. Counting its external calls defines a random variable whose event preimages are visible.
ReAct · From structured outcomes to a numeric variable
Define $X(\omega)$ as the number of external tool calls in trace $\omega$. The illustrated trace has $X=2$.
Use six illustrative traces, each with mass $1/6$. Their call counts are $(1,1,2,2,2,3)$.
$X^{-1}((-\infty,2])=\{T_1,T_2,T_3,T_4,T_5\}$
Thus $F_X(2)=5/6$.
Conditioning on the four successful traces restricts the outcome set and gives each retained trace mass $1/4$.
ReAct · Calculate probabilities by selecting records
| Trace | External calls | $X$ | Solved? |
|---|---|---|---|
| T1 | Search | 1 | No |
| T2 | Search | 1 | Yes |
| T3 | Search, Lookup | 2 | Yes |
| T4 | Search, Search | 2 | No |
| T5 | Search, Lookup | 2 | Yes |
| T6 | Search, Lookup, Search | 3 | Yes |
Search and Lookup count toward $X$. Finish returns the answer.
Uniform mass on the selected records.
Each scenario hides a mathematical structure from this lecture. Identify the structure, justify it, and work through the resulting problem.
A model server's latency follows a known continuous distribution. Rather than sorting a huge pile of requests, you want to compute the deadline that 95% of requests will meet.
Latency $T$ has density $f_T(t)=\frac12e^{-t/2}$ for $t\ge0$ and zero otherwise.
Under normal operation, a detector observes a feature uniformly between $-1$ and $1$, then squares it to produce an anomaly score. Squaring folds the negative and positive halves together—what distribution does that create?
$X\sim\mathrm{Uniform}[-1,1]$ and $Y=X^2$.
Uniformly choose one of 100 labelled records. Let $Y\in\{0,1\}$ be the true label and $\widehat{Y}\in\{0,1\}$ the weak annotator's label. The table gives record counts.
$\begin{array}{c|cc} & \widehat{Y}=1 & \widehat{Y}=0 \\ \hline Y=1 & 18 & 2 \\ Y=0 & 8 & 72 \end{array}$