Course Description
This course is intended for graduate students who plan to explore machine learning further. It will focus on the mathematical and computational foundations necessary for AI research, including linear algebra, calculus, and probability theory. We will adopt a first-principles approach in this course and build everything from very primitive mathematical concepts. Depending on your background, the course material might be a recap or new.
Communication
All written communication should be directed through Piazza. Sign-up instructions will be sent to your email. You can post publicly or privately, depending on your preference. Emails won’t be responded to. Except for the sign-up phase of the class, we will not be using D2L for anything else.
Course Syllabus and Policy
Details on course policies can be found here.
Mathematical Notation
A description of the mathematical notation used in this class can be found here.
Schedule and Syllabus
| Date | Slides |
|---|---|
| Mon Jan 13 | Introduction |
| Linear Algebra | |
| Wed Jan 15 | Vector Spaces, Basis, and Dimension, Direct Sum: lecture notes |
| Mon Jan 20 | No Class (University Holiday) |
| Wed Jan 22 | Linear Maps, Matrices, Invertible Maps and Matrices: lecture notes |
| Mon Jan 27 | Transpose, Change of Basis, Rank of a Matrix, Determinant: lecture notes |
| Wed Jan 29 | Eigenvalues, Characteristic Polynomial, Trace of Matrix: lecture notes |
| Mon Feb 03 | Diagonalization, Triangular Matrices, Metric Spaces, Normed Spaces;p-norms: lecture notes |
| Wed Feb 05 | Norms on $\mathbb{R}^d$ are Equivalent, Convex Set Induces a Norm, Spaces of continuous and differentiable functions: lecture notes |
| Mon Feb 10 | Inner Product and Hilbert Spaces, Orthogonal Vectors and Basis, Orthogonal Matrices: lecture notes |
| Wed Feb 12 | Symmetric Matrices, Spectral Theorem for Symmetric Matrices, Positive Definite Matrices, Variational Characterization of Eigenvalues: lecture notes |
| Mon Feb 17 | Singular Value Decomposition, Matrix Norms, Rank-K Approximation, Pseudo-Inverse of a Matrix: lecture notes |
| Calculus | |
| Wed Feb 19 | Sequences and Convergence, Continuity, Sequence of Functions; Pointwise and Uniform Convergence: lecture notes |
| Mon Feb 24 | Differentiation on $\mathbb{R}$, Riemann Integral on $\mathbb{R}$, Fundamental Theorem of Calculus on $\mathbb{R}$: lecture notes |
| Wed Feb 26 | Mid-Term Exam |
| Mon Mar 03 | No Class (fall break) |
| Wed Mar 05 | No Class (fall break) |
| Mon Mar 10 | Power Series, Taylor Series: lecture notes |
| Wed Mar 12 | $\sigma$-Algebra, Measure: lecture notes |
| Mon Mar 17 | Differentiation on $\mathbb{R}^n$: partial, total, and directional derivatives: lecture notes |
| Wed Mar 19 | Differentiation on $\mathbb{R}^n$: higher-order derivatives, Minima, maxima, saddle points, Matrix calculus: lecture notes |
| Probability Theory | |
| Mon Mar 24 | Definition of a probability measure, Different types of measures; discrete, with density; Radon-Nikodym: lecture notes |
| Wed Mar 26 | Different types of measures: singular measures, Lebesgue decomposition, CDF, Random variables, Conditional probabilities: lecture notes |
| Mon Mar 31 | Bayes Theorem, Independence, Expectation (discrete case), Variance, covariance, correlation (discrete case): lecture notes |
| Wed Apr 02 | Expectation and covariance (general case), Markov and Chebyshev’s Inequality, Example distributions: binomial, Poisson, multivariate normal: lecture notes |
| Mon Apr 07 | Convergence of Random Variables, Borel-Cantelli: lecture notes |
| Wed Apr 09 | Law of Large Numbers; Central Limit Theorem, Concentration Inequalities: lecture notes |
| Mon Apr 14 | Product space and joint distribution, Marginal distribution, Conditional Distribution, Conditional Expectation: lecture notes |
| Wed Apr 16 | Final Exam |
| Mon Apr 21 | No Class (instructor at conference) |
| Mon Apr 23 | No Class (instructor at conference) |