Introduction to Machine Learning

Fall 2024

Welcome to CS 189/289A! This class covers theoretical foundations, algorithms, methodologies, and applications for machine learning. Topics may include supervised methods for regression and classication (linear models, trees, neural networks, ensemble methods, instance-based methods); generative and discriminative probabilistic models; deep learning models including CNNs, transformers, graph neural networks for vision and language tasks; and Markovian models for reinforcement learning and robotics.

Textbook: Deep Learning by Bishop and Bishop.

Professors will post slides prior to lecture at this Google Drive folder (for faster access). The material here is redundant with the website, but it may take up to a day or two for the website to get updated with the slides after lecture. The “Post-Lecture” subfolder contains updates to slides that the professors may make right after lecture.

Note: The topics for future lectures, discussions, and HWs are tentative and may be moved around, changed, or removed.

Overview

Week1

8/29
	LECTURE 1 Introduction and Logistics	[PDF] [VEDIO]
	HOMEWORK 1 Math Review (Due 9/11 11:59pm)	[PDF] [ZIP]

Week2

9/3	LECTURE 2 Maximum Likelihood Estimation	[PDF] [VEDIO] [EXTRA]
	DISCUSSION 0 Math Pre-Requisites Review
9/5	LECTURE 3 Multivariate Gaussians	[PDF] [VEDIO]

Week3

9/10	LECTURE 4 Linear Regression 1	[PDF] [VEDIO]
	DISCUSSION 1 MLE & Gaussians
9/12	LECTURE 5 Linear Regression 2	[PDF] [VEDIO] [EXTRA]
	HOMEWORK 2 Linear/Logistic Regression & Classification (Due 9/25 11:59pm)	[PDF]

Week4

9/17	LECTURE 6 Classification - Generative & Discriminative	[PDF] [VEDIO]
	DISCUSSION 2 Linear Regression
9/19	LECTURE 7 Logistic Regression & Neural Networks	[PDF] [VEDIO] [EXTRA]

Week5

9/24	LECTURE 8 Backpropagation and Gradient Descent 1	[PDF] [VEDIO] [EXTRA]
	DISCUSSION 3 Classification & Logistic Regression
9/26	LECTURE 9 Backpropagation and Gradient Descent 2	[PDF] [VEDIO]
	HOMEWORK 3 Neural Networks (Due 10/9 11:59pm)	[PDF] [ZIP]

Week6

10/1	LECTURE 10 Neural Networks - CNNs, Batch Norm, & ResNets	[PDF] [VEDIO]
	DISCUSSION 4 Neural Networks and Training
10/3	LECTURE 11 Neural Networks - Attention & Transformers	[PDF] [VEDIO] [EXTRA]

Week7

10/8	LECTURE 12 Dimensionality Reduction & PCA	[PDF] [VEDIO] [EXTRA]
	DISCUSSION 5 Convolution and Attention
10/10	LECTURE 13 t-SNE	[PDF] [VEDIO] [EXTRA]
	HOMEWORK 4 Dimensionality Reduction & Decision Theory (Due 10/23 11:59pm)	[PDF] [ZIP]

Week8

10/15	LECTURE 14 Clustering	[PDF] [VEDIO]
	DISCUSSION 6 Dimensionality Reduction Techniques
10/16	Midterm
10/17	LECTURE 15 Multiway Classification, Decision Theory, & Model Evaluation	[PDF] [VEDIO] [EXTRA]

Week9

10/22	LECTURE 16 Nearest Neighbors & Metric Learning	[PDF] [VEDIO]
	DISCUSSION 7 Clustering and Decision Theory
10/24	LECTURE 17 Decision Trees & Ensembling	[PDF] [VEDIO] [VEDIO_EXTRA] [EXTRA]
	HOMEWORK 5 Bias/Variance, Nearest Neighbors, Decision Trees (Due 11/6 11:59pm)	[PDF] [ZIP]

Week10

10/29	LECTURE 18 Bias-Variance Tradeoff & Over/Under-Fitting	[PDF] [VEDIO] [EXTRA]
	DISCUSSION 8 Bias/Variance and Nearest Neighbors
10/31	LECTURE 19 Hidden Markov Models & Graphical Models 1

Week11

11/5	LECTURE 20 Hidden Markov Models & Graphical Models 2	[PDF] [VEDIO] [VEDIO_EXTRA] [EXTRA]
	DISCUSSION 9 Decision Trees and HMMs Intro
11/7	LECTURE 21 Markov Decision Processes	[PDF] [VEDIO]
	HOMEWORK 6 Markovian Models & Reinforcement Learning (Due 11/20 11:59pm)	[PDF] [ZIP]

Week12

11/12	LECTURE 22 Reinforcement Learning
	DISCUSSION 10 HMMs Advanced and MDPs
11/14	LECTURE 23 Robotics and Machine Learning

Week13

11/19	LECTURE 24 Graph Neural Networks & Rotational Equivariance 1
	DISCUSSION 11 MDPs & Reinforcement Learning
11/21	LECTURE 25 Graph Neural Networks & Rotational Equivariance 2
	HOMEWORK 7 Graph Neural Networks & Applications of Deep Learning (Due 12/4 11:59pm)

---

Notes

• Note 2- Linear Regression
• Note 3- Feature Engineering
• Note 4- MLE and MAP for Regression
• Note 5- Bias-Variance Tradeoff
• Note 6- Multivariate Gaussians
• Note 7- MLE and MAP Part 2
• Note 8- Kernels and Ridge Regression
• Note 9- Total Least Squares
• Note 10- PCA
• Note 11- CCA
• Note 12- Optimization
• Note 13- Gradient Descent
• Note 14- Neural Networks
• Note 15- Training Neural Networks
• Note 16- Generative vs. Discriminative Classification
• Note 18- Gaussian Discriminant Analysis
• Note 19- Clustering
• Note 20- Support Vector Machines
• Note 21- Generalization and Stability
• Note 22- Duality
• Note 25- Decision Trees
• Note 26- Boosting
• Note 27- Convolutional Neural Networks
• machlearn
• math4ml