| 0 | Kick-off | Slides | — | The course in one picture | — |
|---|
| 1 | History of AI and machine learning | Slides | Kolter & Do, Linear Algebra Review & Reference (CS229) §1–2 · Boyd & Vandenberghe, Introduction to Applied Linear Algebra (VMLS) Ch. 1–3 | The Imitation Game · Turing machine — short | — |
|---|
| LA1 | Notation & basic objects; vector/matrix products; properties of matrix multiplication | — | Kolter & Do §1–2 | 3Blue1Brown, Essence of Linear Algebra · Deisenroth, Faisal & Ong, Mathematics for Machine Learning, Ch. 2–4 | — |
|---|
| LA2 | Transpose, symmetric matrices, trace; norms, Cauchy–Schwarz & cosine similarity | — | Kolter & Do §3.2–3.5 | Demo: ℓₚ unit balls · Demo: cosine similarity | — |
|---|
| LA3 | Matrix inverse; span, range & linear independence | — | Kolter & Do §3.6–3.9 | — | — |
|---|
| LA4 | Linear projection & least squares | — | Kolter & Do §3.9, §4.4 | Demo: least-squares line fitting | — |
|---|
| LA5 | Determinant; condition number & numerical sensitivity | — | Kolter & Do §3.10 · condition number is supplementary — see Trefethen & Bau, Numerical Linear Algebra | — | — |
|---|
| LA6 | Quadratic forms; symmetric & positive (semi)definite matrices | — | Kolter & Do §3.11 | — | — |
|---|
| LA7 | Eigenvalues & eigenvectors; diagonalization & the spectral theorem; eigenvalues as optimisation | — | Kolter & Do §3.12–3.13, §4.6 | Demo: eigenvectors as fixed directions · Demo: rotate, scale, rotate · Setosa, Eigenvectors & Eigenvalues, Explained Visually | Practice-1 |
|---|
| LA8 | SVD; PCA; matrix calculus (self study) | — | Kolter & Do §4.1–4.3 | Demo: PCA playground · Setosa, Principal Component Analysis, Explained Visually · Petersen & Pedersen, The Matrix Cookbook | Practice-2 |
|---|
| PR1 | Probability review (self study + extra class) | — | Probability Review | — | — |
|---|
| OPT1 | Why optimise; anatomy of a problem; stationarity & convexity; the Lagrangian (shadow price, scalarisation) | — | Deisenroth, Faisal & Ong, Mathematics for ML Ch. 7 (§7.1–7.3) | Boyd & Vandenberghe, Convex Optimization Ch. 1, §3.1–3.2, §4.7, §5.6 · Khan Academy, Interpretation of Lagrange Multipliers | — |
|---|
| OPT2 | Gradient descent: steepest descent; step size & curvature; descent lemma; condition number & zigzag; line search | — | Dive into Deep Learning §12.3 Gradient Descent · Dive into Deep Learning §12.1–12.2 (optimization & convexity) | Boyd & Vandenberghe §9.2–9.4 (descent methods, backtracking, steepest descent) · Strang, MIT 18.065 Lec. 22: Gradient Descent · Boyd, EE364a slides §9 Unconstrained Minimization | — |
|---|
| OPT3 | Stochastic gradient descent: unbiased minibatch gradients; the √B law; noise floor; step-size schedules | — | Dive into Deep Learning §12.4 SGD · Dive into Deep Learning §12.5 Minibatch SGD · Dive into Deep Learning §12.11 Learning-Rate Scheduling | Bottou, Curtis & Nocedal, Optimization Methods for Large-Scale ML §3.3, §4.2 (Thms 4.6–4.7) · Strang, MIT 18.065 Lec. 25: Stochastic Gradient Descent | — |
|---|
| EBM | Linear classifiers; perceptron, hinge & logistic losses; MSE vs likelihood; NLL as push-down / pull-up | — | LeCun et al., A Tutorial on Energy-Based Learning §2.1, §2.2.1–2.2.4 (NLL gradient: eq. 24) | Goodfellow, Bengio & Courville, Deep Learning §5.5, §5.5.1, §5.7.1 · Bishop, Pattern Recognition and Machine Learning §1.2.5, §3.1.1, §4.3.4 | — |
|---|
| CI | Concentration inequalities: the sample average; Markov, Chebyshev & Hoeffding (with proof); sample complexity; laws of large numbers | — | Abhishek et.al., Introduction to Concentration Inequalities §3.1–3.5 | Chan, Introduction to Probability for Data Science Ch. 6 (§6.2.4–6.2.8, §6.3) · Duchi, CS229 Supplemental Notes: Hoeffding's Inequality | Homework |
|---|