Course Description
This course is intended for graduate students who plan to dive further into machine learning. This course will focus on the mathematical and computational foundations necessary for AI research. This includes linear algebra, calculus, and probability theory. We will adopt a first principles approach in this course and build everything from very primitive mathematical concepts. The course material might be a recap or new, depending on your background.
Communication
All written communication should be directed through Piazza. Sign-up instructions will be sent to your email. You can post publicly or privately, depending on your preference. Emails won’t be responded to. Except for the sign-up phase of the class, we will not be using D2L for anything else.
Assignments
We will be using GitHub Classroom for all the assignments in this course. Sign-up instructions will be sent to your email.
Course Syllabus and Policy
Details on course policies can be found here.
Mathematical Notation
A description of the mathematical notation used in this class can be found here.
Schedule and Syllabus
| Date | Slides |
|---|---|
| Introduction | |
| Linear Algebra | |
| Mon Aug 28 | Vector Spaces, Basis, and Dimension, Direct Sum: lecture notes, scribe notes |
| Wed Aug 30 | Linear Maps, Matrices, Invertible Maps and Matrices: lecture notes, scribe notes |
| Mon Sep 04 | No Class (Labor Day) |
| Wed Sep 06 | Transpose, Change of Basis, Rank of a Matrix, Determinant: lecture notes, scribe notes |
| Mon Sep 11 | Eigenvalues, Characteristic Polynomial, Trace of Matrix: lecture notes, scribe notes |
| Wed Sep 13 | Diagonalization, Triangular Matrices, Metric Spaces, Normed Spaces;p-norms: lecture notes, scribe notes |
| Mon Sep 18 | Norms on $\mathbb{R}^d$ are Equivalent, Convex Set Induces a Norm, Spaces of continuous and differentiable functions: lecture notes, scribe notes |
| Wed Sep 20 | Inner Product and Hilbert Spaces, Orthogonal Vectors and Basis, Orthogonal Matrices: lecture notes, scribe notes |
| Mon Sep 25 | Symmetric Matrices, Spectral Theorem for Symmetric Matrices, Positive Definite Matrices, Variational Characterization of Eigenvalues: lecture notes, scribe notes |
| Wed Sep 27 | No Lecture |
| Mon Oct 02 | Singular Value Decomposition, Matrix Norms, Rank-K Approximation, Pseudo-Inverse of a Matrix: lecture notes, scribe notes |
| Calculus | |
| Wed Oct 04 | Sequences and Convergence, Continuity, Sequence of Functions; Pointwise and Uniform Convergence: lecture notes, scribe notes |
| Mon Oct 09 | Differentiation on $\mathbb{R}$, Riemann Integral on $\mathbb{R}$, Fundamental Theorem of Calculus on $\mathbb{R}$: lecture notes, scribe notes |
| Wed Oct 11 | Power Series, Taylor Series: lecture notes, scribe notes |
| Mon Oct 16 | Mid-term Review |
| Wed Oct 18 | Mid-Term Exam |
| Mon Oct 23 | No Class (fall break) |
| Wed Oct 25 | $\sigma$-Algebra, Measure: lecture notes, scribe notes |
| Mon Oct 30 | Lebesgue Measure on $\mathbb{R}^n$, A set that is not Lebesgue measurable: lecture notes, scribe notes |
| Wed Nov 01 | Lebesgue Integral on $\mathbb{R}^n$, Differentiation on $\mathbb{R}^n$: partial, total, and directional derivatives: lecture notes, scribe notes |
| Mon Nov 06 | Differentiation on $\mathbb{R}^n$: higher-order derivatives, Minima, maxima, saddle points, Matrix calculus: lecture notes, scribe notes |
| Probability Theory | |
| Wed Nov 08 | Definition of a probability measure, Different types of measures; discrete, with density; Radon-Nikodym: lecture notes, scribe notes |
| Mon Nov 13 | Different types of measures: singular measures, Lebesgue decomposition, CDF, Random variables, Conditional probabilities: lecture notes, scribe notes |
| Wed Nov 15 | Bayes Theorem, Independence, Expectation (discrete case), Variance, covariance, correlation (discrete case): lecture notes, scribe notes |
| Mon Nov 20 | Expectation and covariance (general case), Markov and Chebyshev’s Inequality, Example distributions: binomial, Poisson, multivariate normal: lecture notes, scribe notes |
| Wed Nov 22 | Convergence of Random Variables, Borel-Cantelli: lecture notes, scribe notes |
| Mon Nov 27 | Law of Large Numbers; Central Limit Theorem, Concentration Inequalities: lecture notes, scribe notes |
| Wed Nov 29 | Product space and joint distribution, Marginal distribution, Conditional Distribution, Conditional Expectation: lecture notes, scribe notes |
| Mon Dec 04 | Final Review |
| Wed Dec 06 | No-Class |
| Mon Dec 11 | Final Exam (3:00 pm - 5:00 pm) |