Differentiation on $\mathbb{R}$, Riemann Integral, Fundamental Theorem of Calculus


What local signal do modern optimizers start from?

Vishnu Boddeti

Example 1

Adaptive optimizers begin with a noisy local gradient

What local signal do AdamW and matrix-preconditioned optimizers start from?

x₀ x₁ x₂ lower loss
  • Modern optimizers rescale, average, precondition, and regularize gradients, but every update begins with the same local derivative signal.
  • What does a raw gradient step look like on the loss curve, and why do its learning rate and starting point matter?

The gradient descent update

local tangent Δx = -η f'(xₙ) xₙ xₙ₊₁ xₙ₊₁ = xₙ - ηf'(xₙ)
  • Given a differentiable $f:\mathbb{R}\to\mathbb{R}$ and a starting point $x_0$, gradient descent repeats $x_{n+1} = x_n - \eta f'(x_n)$ for a learning rate $\eta > 0$.
  • Since $f'(x_n)$ points in the direction of steepest increase, subtracting a multiple of it moves $x_n$ downhill.
  • Each step follows the local slope only
  • it has no knowledge of the global shape of $f$, which is exactly why the starting point $x_0$ can determine which minimum (if there are several) the sequence converges to.

Set a start point and take one gradient step at a time

$f(x) = x^4 - 3x^2 + x$ drag to set start current point visited path
Compare several start points. Gradient descent uses only the local slope, so different starts can reach different minima.

Example 2

Flow-matching objectives average difficulty across time

What is flow matching?

t = 0 t = 0.5 t = 1 noise p₀ velocity vθ(x,t) data p₁
  • Choose a probability path that gradually transforms simple noise into data.
  • A neural network $v_\theta(x,t)$ predicts the instantaneous velocity at location $x$ and time $t$.
  • Training matches that prediction to a target velocity $u_t(x)$ along the path.

Why does a time integral appear?

0 time t 1 sampled times in a minibatch average finite samples ≈ integrate losses over all t
  • Every time $t$ has a velocity-matching loss $\ell_\theta(t)$.
  • The population objective is $\mathcal{L}_{\mathrm{FM}}(\theta)=\int_0^1 \ell_\theta(t)\,dt$.
  • Sampling $t\sim\mathrm{Uniform}(0,1)$ gives a finite stochastic estimate of that integral.
  • A Riemann sum makes the same connection by partitioning time into narrow bins.

Why do diffusion and flow-matching objectives integrate loss over noise level or time?

∫₀¹ w(t)L(t) dt weighted loss noise time t 0
  • A diffusion or flow model is trained at continuously many noise levels or times, but a minibatch samples only finitely many.
  • The population objective is an integral over time; the training estimate is a finite weighted sum.
  • How does that sum approximate the continuous objective?

Riemann sums approximate the integral

L(t) 0 1 Sₙ → ∫₀¹ L(t) dt
  • Partition $[a,b]$ into $n$ equal subintervals of width $\Delta x = (b-a)/n$ and sum $f$ evaluated at a sample point in each: $S_n = \sum_{i=1}^n f(x_i^*)\,\Delta x$.
  • As $n \to \infty$, $S_n \to \int_a^b f(x)\,dx$, the exact area under the curve.
  • The Fundamental Theorem of Calculus says this limit equals $F(b) - F(a)$ for any antiderivative $F$ with $F' = f$
  • connecting the "sum of infinitely many thin rectangles" picture directly back to differentiation.

Increase sampled time bins and compare the training sum with the integral

$f(x) = \sin(x) + 2$ midpoint rectangles
Each rectangle is a time or noise-level bin. As the bins narrow, the finite training sum approaches the continuous-time objective.

Example 3

A sampler discretizes continuous generative dynamics

What is an ordinary differential equation?

x(0): noise x(1): sample dx/dt = vθ(x(t), t)
  • An ODE specifies a rate of change, not the whole path in advance.
  • Given the starting noise $x(0)$, the velocity field determines the continuous trajectory $x(t)$.
  • The probability-flow ODE is deterministic: the same initial noise and model produce the same path.

How do we discretize the ODE?

continuous path Euler steps t₀ t₁ t₂ t₃
  • Choose times $t_k=k\Delta t$ rather than evaluating every instant.
  • Euler's method freezes the local velocity for one step: $x_{k+1}=x_k+\Delta t\,v_\theta(x_k,t_k)$.
  • Smaller $\Delta t$ usually follows the continuous path more closely, but requires more model evaluations.

How does a continuous probability flow become a finite-step sampler?

  • Flow and diffusion samplers numerically integrate a learned time-dependent vector field.
  • More solver steps usually reduce discretization error, but each additional model evaluation increases latency.

Increase solver steps and compare trajectory accuracy with inference cost

Move the control to compare the two normalized responses.
Modern samplers search for the fewest evaluations that preserve the learned continuous trajectory well enough.

Practice problems

Each scenario hides a mathematical structure from this lecture. Identify the structure, justify it, and work through the resulting problem.

Problem 1 Hyperparameter optimization

Tune a scalar regularization strength

A single regularization dial trades underfitting against instability: turn it too far either way and validation loss rises. Where should the dial be set, and how do you know the choice is globally best?

Model evidence

$L(\lambda)=\lambda+4/\lambda$ for $\lambda>0$.

  1. Which positive settings are stationary?
  2. Which setting minimizes the loss, and what is the minimum value?
  3. What does each term in the objective do on the two sides of the optimum?
Problem 2 Monitoring

Recover cumulative deployment harm from its rate

A monitor reports the instantaneous rate at which harmful predictions accumulate as traffic grows. An auditor asks a more concrete question: how much harm has accumulated in total since deployment began?

Model evidence

The rate is $r(t)=3t^2+2$ harmful-score units per hour on $0\le t\le T$.

  1. What cumulative-harm function starts at zero and has the reported rate?
  2. Does differentiating your answer recover the monitor's rate?
  3. How much harm accumulates during the first two hours?
Problem 3 Calibration

Measure area under a calibration error curve

Calibration error changes sign at one confidence value. If you integrate the signed expression blindly, positive and negative errors cancel even though absolute error cannot. Where should the calculation split?

Model evidence

$e(p)=|2p-1|$ for $p\in[0,1]$.

  1. At which confidence does the formula change, and what integral represents the absolute error correctly?
  2. What is the average error when confidence is uniformly distributed?
  3. Why would ignoring the change point give a misleading cancellation?