Course Description
This course is intended for graduate students who plan to explore machine learning further. It will focus on the mathematical and computational foundations necessary for AI research, including linear algebra, calculus, and probability theory. We will adopt a first-principles approach in this course and build everything from very primitive mathematical concepts. Depending on your background, the course material might be a recap or new.
Communication
All written communication should be directed through Piazza. Sign-up instructions will be sent to your email. You can post publicly or privately, depending on your preference. Emails won’t be responded to. Except for the sign-up phase of the class, we will not be using D2L for anything else.
Course Syllabus and Policy
Details on course policies can be found here.
Mathematical Notation
A description of the mathematical notation used in this class can be found here.
Schedule and Syllabus
| Date | Slides |
|---|---|
| Mon Aug 31 | Introduction |
| Linear Algebra | |
| Wed Sep 02 | Vector Spaces, Basis, and Dimension, Direct Sum lecture notes demo |
| Mon Sep 07 | No Class (Labor Day) |
| Wed Sep 09 | Linear Maps, Matrices, Invertible Maps and Matrices lecture notes demo |
| Mon Sep 14 | Transpose, Change of Basis, Rank of a Matrix, Determinant lecture notes demo |
| Wed Sep 16 | Eigenvalues, Characteristic Polynomial, Trace of Matrix lecture notes demo |
| Mon Sep 21 | Diagonalization, Triangular Matrices, Metric Spaces, Normed Spaces;p-norms lecture notes |
| Wed Sep 23 | Norms on $\mathbb{R}^d$ are Equivalent, Convex Set Induces a Norm, Spaces of continuous and differentiable functions lecture notes |
| Mon Sep 28 | Inner Product and Hilbert Spaces, Orthogonal Vectors and Basis, Orthogonal Matrices lecture notes |
| Wed Sep 30 | Symmetric Matrices, Spectral Theorem for Symmetric Matrices, Positive Definite Matrices, Variational Characterization of Eigenvalues lecture notes |
| Mon Oct 05 | Singular Value Decomposition, Matrix Norms, Rank-K Approximation, Pseudo-Inverse of a Matrix lecture notes |
| Wed Oct 07 | Mid-Term Exam Review |
| Mon Oct 12 | Mid-Term Exam (in person) |
| Calculus | |
| Wed Oct 14 | Sequences and Convergence, Continuity, Sequence of Functions; Pointwise and Uniform Convergence lecture notes |
| Mon Oct 19 | Differentiation on $\mathbb{R}$, Riemann Integral on $\mathbb{R}$, Fundamental Theorem of Calculus on $\mathbb{R}$ lecture notes |
| Wed Oct 21 | Power Series, Taylor Series lecture notes |
| Mon Oct 26 | No Class (fall break) |
| Wed Oct 28 | No Class |
| Mon Nov 02 | Differentiation on $\mathbb{R}^n$: partial, total, and directional derivatives lecture notes |
| Wed Nov 04 | Differentiation on $\mathbb{R}^n$: higher-order derivatives, Minima, maxima, saddle points, Matrix calculus lecture notes |
| Probability Theory | |
| Mon Nov 09 | Probability spaces and axioms; discrete distributions and densities lecture notes |
| Wed Nov 11 | Cumulative distribution functions, Random variables, Conditional probability lecture notes |
| Mon Nov 16 | Bayes Theorem, Independence, Expectation (discrete case), Variance, covariance, and correlation (discrete case) lecture notes |
| Wed Nov 18 | Expectation and covariance (general case), Markov and Chebyshev’s Inequality, Example distributions: binomial, Poisson, multivariate normal lecture notes |
| Mon Nov 23 | Convergence of Random Variables, Borel-Cantelli lecture notes |
| Wed Nov 25 | Law of Large Numbers; Central Limit Theorem, Concentration Inequalities lecture notes |
| Mon Nov 30 | Joint distributions, Marginal distributions, Conditional distributions, Conditional Expectation lecture notes |
| Wed Dec 02 | Final Exam Review |
| Mon Dec 07 | Final Exam (in person) |
| Wed Dec 09 | No Class (final exam completed) |