Course Format

Students are expected to listen to pre-recorded lectures before coming to class. The class hours will be utilized for a recap and answering any questions.

Course Topics

• Linear Algebra
  • Vector Space
  • Basis and Dimension
  • Direct Sum
  • Linear Maps, Kernel, Range
  • Matrices
  • Invertible Maps and Matrices
  • Transpose
  • Change of Basis
  • Rank of a Matrix
  • Determinant
  • Eigenvalues
  • Characteristic Polynomial
  • Trace of Matrix
  • Diagonalization; Triangular Matrices
  • Metric Spaces
  • Normed Spaces; p-norms
  • Norms on $\mathbb{R}^d$ are Equivalent
  • Convex Set Induces a Norm
  • Spaces of continuous and differentiable functions
  • Inner Product and Hilbert Spaces
  • Orthogonal Vectors and Basis
  • Orthogonal Matrices
  • Symmetric Matrices
  • Spectral Theorem for Symmetric Matrices
  • Positive Definite Matrices
  • Variational Characterization of Eigenvalues
  • Singular Value Decomposition
  • Matric Norms; Rank-K Approximation;
  • Pseudo-Inverse of a Matrix
• Calculus
  • Sequences and Convergence
  • Continuity
  • Sequence of Functions, Pointwise and Uniform Convergence
  • Differentiation on $\mathbb{R}$
  • Riemann Integral on $\mathbb{R}$
  • Fundamental Theorem of Calculus on $\mathbb{R}$
  • Power Series
  • Taylor Series
  • $\sigma$-Algebra
  • Measure
  • Lebesgue Measure on $\mathbb{R}^n$
  • A set that is not Lebesgue measurable
  • Lebesgue Integral on $\mathbb{R}^n$
  • Differentiation on $\mathbb{R}^n$; partial derivatives
  • Differentiation on $\mathbb{R}^n$; total derivative
  • Differentiation on $\mathbb{R}^n$; directional derivative
  • Differentiation on $\mathbb{R}^n$; higher-order derivatives
  • Minima, maxima, saddle
  • Matrix Calculus
• Probability Theory
  • Definition of a probability measure
  • Different types of measures; discrete, with density; Radon-Nikodym
  • Different types of measures; singular measures, Lebesgue decomposition
  • Cumulative distribution function
  • Random variables
  • Conditional probabilities
  • Bayes Theorem
  • Independence
  • Expectation (discrete case)
  • Variance, covariance, correlation (discrete case)
  • Expectation and covariance (general case)
  • Markov and Chebyshev’s Inequality
  • Example distributions; binomial, Poisson, multivariate normal
  • Convergence of Random Variables
  • Borel-Cantelli
  • Law of Large Numbers; Central Limit Theorem
  • Concentration Inequalities
  • Gilvenko-Cantelly Theorem
  • Product space and joint distribution
  • Marginal Distribution
  • Conditional Distribution
  • Conditional Expectation
• Optional Topics
  • Statistics
    • Estimation, bias, variance
    • Confidence sets
    • Maximum likelihood estimator
    • Sufficiency, identifiability
    • Hypothesis testing, level and power of a test
    • Likelihood-ratio tests and Neyman-Pearson Lemma
    • $p$-values
    • Multiple testing
    • Non-parametric tests (rank and permutation tests, Kolmogorov-Smirnov)
    • The bootstrap
    • Bayesian Statistics
  • Assorted Topics
    • High-dimensional spaces
    • What is a convex optimization problem?
    • Convex optimization, Lagrangian, the dual problem

Assignments and Grading

• Four long-form homeworks: 30%
• Four multiple-choice homeworks: 20%
• Mid-term exam: 20%
• Final exam: 20%
• Lecture Scribe: 10%
  • Each student is required to participate in scribing one lecture using LaTeX. The template can be downloaded at Scribe Template.
  • 2-3 students can be involved in scribing a lecture together. To ensure that the notes will be available for students currently in the course, scribed notes are due within one week after the lecture (no late submissions accepted). Please use this Google Doc to reserve lectures for scribing.

Grading Rubric

Late Days

• 5 free late days total (not per assignment)
• beyond that, a 10% reduction of points per late-day
• use free late days wisely, and save them for assignments towards the end

Plagiarism

This course has adopted the Chegg and Similar Sites policy. Submission of student work (e.g., assignments and/or exam solutions) based on those found on Chegg, Brainly, Quizlet, and other similar websites will result in an Academic Dishonesty Report (ADR) and an automatic failing grade of zero (0.0) for the course. The ADR for students personally posting questions from assignments or exams to these sites will request additional sanctions.