Course Description
This course is intended for graduate students who plan to explore machine learning further. It will focus on the mathematical and computational foundations necessary for AI research, including linear algebra, calculus, and probability theory. We will adopt a first-principles approach in this course and build everything from very primitive mathematical concepts. Depending on your background, the course material might be a recap or new.
Communication
All written communication should be directed through Piazza. Sign-up instructions will be sent to your email. You can post publicly or privately, depending on your preference. Emails won’t be responded to. Except for the sign-up phase of the class, we will not be using D2L for anything else.
Course Syllabus and Policy
Details on course policies can be found here.
Mathematical Notation
A description of the mathematical notation used in this class can be found here.
Schedule and Syllabus
| Date | Slides |
|---|---|
| Mon Aug 25 | Introduction |
| Linear Algebra | |
| Wed Aug 27 | Vector Spaces, Basis, and Dimension, Direct Sum lecture notes scribe notes |
| Mon Sep 01 | No Class (Labor Day) |
| Wed Sep 03 | Linear Maps, Matrices, Invertible Maps and Matrices lecture notes scribe notes |
| Mon Sep 08 | Transpose, Change of Basis, Rank of a Matrix, Determinant lecture notes scribe notes |
| Wed Sep 10 | Eigenvalues, Characteristic Polynomial, Trace of Matrix lecture notes scribe notes |
| Mon Sep 15 | Diagonalization, Triangular Matrices, Metric Spaces, Normed Spaces;p-norms lecture notes scribe notes |
| Wed Sep 17 | Norms on $\mathbb{R}^d$ are Equivalent, Convex Set Induces a Norm, Spaces of continuous and differentiable functions lecture notes scribe notes |
| Mon Sep 22 | Inner Product and Hilbert Spaces, Orthogonal Vectors and Basis, Orthogonal Matrices lecture notes scribe notes |
| Wed Sep 24 | Symmetric Matrices, Spectral Theorem for Symmetric Matrices, Positive Definite Matrices, Variational Characterization of Eigenvalues lecture notes scribe notes |
| Mon Sep 29 | Singular Value Decomposition, Matrix Norms, Rank-K Approximation, Pseudo-Inverse of a Matrix lecture notes scribe notes |
| Wed Oct 01 | Mid-Term Exam Review |
| Mon Oct 06 | Mid-Term Exam |
| Calculus | |
| Mon Oct 13 | Differentiation on $\mathbb{R}$, Riemann Integral on $\mathbb{R}$, Fundamental Theorem of Calculus on $\mathbb{R}$ lecture notes scribe notes |
| Wed Oct 15 | $\sigma$-Algebra, Measure lecture notes scribe notes |
| Mon Oct 20 | No Class (fall break) |
| Wed Oct 22 | Differentiation on $\mathbb{R}^n$: partial, total, and directional derivatives lecture notes scribe notes |
| Mon Oct 27 | Differentiation on $\mathbb{R}^n$: higher-order derivatives, Minima, maxima, saddle points, Matrix calculus lecture notes scribe notes |
| Probability Theory | |
| Wed Oct 29 | Definition of a probability measure, Different types of measures; discrete, with density; Radon-Nikodym lecture notes scribe notes |
| Mon Nov 03 | Different types of measures: singular measures, Lebesgue decomposition, CDF, Random variables, Conditional probabilities lecture notes scribe notes |
| Wed Nov 05 | Bayes Theorem, Independence, Expectation (discrete case), Variance, covariance, correlation (discrete case) lecture notes scribe notes |
| Mon Nov 10 | Expectation and covariance (general case), Markov and Chebyshev’s Inequality, Example distributions: binomial, Poisson, multivariate normal lecture notes scribe notes |
| Wed Nov 12 | Convergence of Random Variables, Borel-Cantelli lecture notes scribe notes |
| Mon Nov 17 | Law of Large Numbers; Central Limit Theorem, Concentration Inequalities lecture notes scribe notes |
| Wed Nov 19 | Product space and joint distribution, Marginal distribution, Conditional Distribution, Conditional Expectation lecture notes scribe notes |
| Mon Nov 24 | Final Exam |
| Wed Nov 26 | ** No Class ** |
| Mon Dec 01 | ** No Class (conference) ** |
| Wed Dec 03 | ** No Class (conference) ** |