ECE 595ML: Machine Learning I

By Stanley H. Chan

Electrical and Computer Engineering, Purdue University, West Lafayette, IN

View Courses

Audio podcast
Video podcast
Slides/Notes podcast

• About
• Questions
• Reviews

Lecture Number/Topic	Online Lecture	Video	Lecture Notes	Supplemental Material	Suggested Exercises
ECE 595ML: Introduction	View on YouTube
ECE 595ML: Course Overview	View HTML	View		ECE 595ML: Course Overview
ECE 595ML Lecture 1.1: Linear Regression	View HTML	View		ECE 595ML Lecture 1.1: Linear Regression
ECE 595ML Lecture 1.2: Linear Regression - Geometry	View HTML	View		ECE 595ML Lecture 1.2: Linear Regression - Geometry
ECE 595ML Lecture 2.1: Regularized Linear Regression	View HTML	View		ECE 595ML Lecture 2.1: Regularized Linear Regression
ECE 595ML Lecture 2.2: Regularized Linear Regression - LASSO Regression	View HTML	View		ECE 595ML Lecture 2.2: Regularized Linear Regression - LASSO Regression
ECE 595ML Lecture 3.1: Linear Regression with Kernels - Kernel Method	View HTML	View		ECE 595ML Lecture 3.1: Linear Regression with Kernals - Kernal Method
ECE 595ML Lecture 3.2: Linear Regression with Kernels - Examples	View HTML	View		ECE 595ML Lecture 3.2: Linear Regression with Kernals - Examples
ECE 595ML Lecture 4.1: Introduction to Optimization - Unconstrained Optimization	View HTML	View		ECE 595ML Lecture 4.1: Introduction to Optimization - Unconstrained Optimization
ECE 595ML Lecture 4.2: Introduction to Optimization - Convexity	View HTML	View		ECE 595ML Lecture 4.2: Introduction to Optimization - Convexity
ECE 595ML Lecture 4.3: Introduction to Optimization - Constrained Optimization	View HTML	View		ECE 595ML Lecture 4.3: Introduction to Optimization - Constrained Optimization
ECE 595ML Lecture 5.1: Gradient Descent	View HTML	View		ECE 595ML Lecture 5.1: Gradient Descent
ECE 595ML Lecture 5.2: Gradient Descent - Stochastic Gradient Descent	View HTML	View		ECE 595ML Lecture 5.2: Gradient Descent - Stochastic Gradient Descent
ECE 595ML Lecture 6.1: Linear Separatability - Notations	View HTML	View		ECE 595ML Lecture 6.1: Linear Separatability - Notations
ECE 595ML Lecture 6.2: Linear Separatability - Geometry of Discriminant Function	View HTML	View		ECE 595ML Lecture 6.2: Linear Separatability - Geometry of Discriminant Function
ECE 595ML Lecture 6.3: Linear Separatability	View HTML	View		ECE 595ML Lecture 6.3: Linear Separatability
ECE 595ML Lecture 7.1: Feature Analysis via PCA	View HTML	View		ECE 595ML Lecture 7.1: Feature Analysis via PCA
ECE 595ML Lecture 7.2: Feature Analysis via PCA - Kernal PCA	View HTML	View		ECE 595ML Lecture 7.2: Feature Analysis via PCA - Kernal PCA
ECE 595ML Lecture 8.1: Hand-Crafted and Deep Features - Convolution	View HTML	View		ECE 595ML Lecture 8.1: Hand-Crafted and Deep Features - Convolution
ECE 595ML Lecture 8.2: Hand-Crafted and Deep Features - SIFT and HOG	View HTML	View		ECE 595ML Lecture 8.2: Hand-Crafted and Deep Features - SIFT and HOG
ECE 595ML Lecture 8.3: Hand-Crafted and Deep Features - Deep Features	View HTML	View		ECE 595ML Lecture 8.3: Hand-Crafted and Deep Features - Deep Features
ECE 595ML Lecture 9.1: Bayesian Decision - Review of High-Dimensional Gaussian	View HTML	View		ECE 595ML Lecture 9.1: Bayesian Decision - Review of High-Dimensional Gaussian
ECE 595ML Lecture 9.2: Bayesian Decision - Basic Principle	View HTML	View		ECE 595ML Lecture 9.2: Bayesian Decision - Basic Principle
ECE 595ML Lecture 9.3: Bayesian Decision - The Three Cases	View HTML	View		ECE 595ML Lecture 9.3: Bayesian Decision - The Three Cases
ECE 595ML Lecture 10: Minimum Probability of Error Rule	View on YouTube
ECE 595ML Lecture 11.1: Maximum-Likelihood Estimation - Basic Principles	View HTML	View		ECE 595ML Lecture 11.1: Maximum-Likelihood Estimation - Basic Principles
ECE 595ML Lecture 11.2: Maximum-Likelihood Estimation - Examples	View HTML	View		ECE 595ML Lecture 11.2: Maximum-Likelihood Estimation - Examples
ECE 595ML Lecture 12.1: Bayesian Parameter Estimation - Basic Principles	View HTML	View		ECE 595ML Lecture 12.1: Bayesian Parameter Estimation - Basic Principles
ECE 595ML Lecture 12.2: Bayesian Parameter Estimation - Choosing Priors	View HTML	View		ECE 595ML Lecture 12.2: Bayesian Parameter Estimation - Choosing Priors
ECE 595ML Lecture 13.1: Connecting Bayesian with Linear Regression - Linear Regression Review	View HTML	View		ECE 595ML Lecture 13.1: Connecting Bayesian with Linear Regression - Linear Regression Review
ECE 595ML Lecture 13.2: Connecting Bayesian with Linear Regression	View HTML	View		ECE 595ML Lecture 13.2: Connecting Bayesian with Linear Regression
ECE 595ML Lecture 14.1: Logistic Regression - From Linear to Logistic	View HTML	View		ECE 595ML Lecture 14.1: Logistic Regression - From Linear to Logistic
ECE 595ML Lecture 14.2: Logistic Regression - Interpreting Logistic	View HTML	View		ECE 595ML Lecture 14.2: Logistic Regression - Interpreting Logistic
ECE 595ML Lecture 14.3: Logistic Regression - Convexity	View HTML	View		ECE 595ML Lecture 14.3: Logistic Regression - Convexity
ECE 595ML Lecture 15.1: Logistic Regression - Gradient Descent	View HTML	View		ECE 595ML Lecture 15.1: Logistic Regression - Gradient Descent
ECE 595ML Lecture 15.2: Logistic Regression - Connection with Bayes	View HTML	View		ECE 595ML Lecture 15.2: Logistic Regression - Connection with Bayes
ECE 595ML Lecture 15.3: Logistic Regression - Comparison with Linear Regression	View HTML	View		ECE 595ML Lecture 15.3: Logistic Regression - Comparison with Linear Regression
ECE 595ML Lecture 16.1: Preceptron - From Logistic to Preceptron	View HTML	View		ECE 595ML Lecture 16.1: Preceptron - From Logistic to Preceptron
ECE 595ML Lecture 16.2: Preceptron - Properties of Preceptron Loss	View HTML	View		ECE 595ML Lecture 16.2: Preceptron - Properties of Preceptron Loss
ECE 595ML Lecture 17.1: Perceptron - Perceptron Algorithm	View HTML	View		ECE 595ML Lecture 17.1: Perceptron - Perceptron Algorithm
ECE 595ML Lecture 17.2: Perceptron - Optimality	View HTML	View		ECE 595ML Lecture 17.2: Perceptron - Optimality
ECE 595ML Lecture 17.3: Perceptron - Convergence	View HTML	View		ECE 595ML Lecture 17.3: Perceptron - Convergence
ECE 595ML Lecture 18.1: Multi-Layer Perceptron	View HTML	View		ECE 595ML Lecture 18.1: Multi-Layer Perceptron
ECE 595ML Lecture 18.2: Multi-Layer Perceptron - Back Propagation	View HTML	View		ECE 595ML Lecture 18.2: Multi-Layer Perceptron - Back Propagation
ECE 595ML Lecture 19.1: Support Vector Machine - Concept of Margin	View HTML	View		ECE 595ML Lecture 19.1: Support Vector Machine - Concept of Margin
ECE 595ML Lecture 19.2: Support Vector Machine - SVM	View HTML	View		ECE 595ML Lecture 19.2: Support Vector Machine - SVM
ECE 595ML Lecture 20.1: Support Vector Machine - Lagrange Duality	View HTML	View		ECE 595ML Lecture 20.1: Support Vector Machine - Lagrange Duality
ECE 595ML Lecture 20.2: Support Vector Machine - Dual SVM	View HTML	View		ECE 595ML Lecture 20.2: Support Vector Machine - Dual SVM
ECE 595ML Lecture 21.1: Support Vector Machine - Soft SVM	View HTML	View		ECE 595ML Lecture 21.1: Support Vector Machine - Soft SVM
ECE 595ML Lecture 21.2: Support Vector Machine - Kernel Trick	View HTML	View		ECE 595ML Lecture 21.2: Support Vector Machine - Kernel Trick
ECE 595ML Lecture 22.1: Is Learning Feasible? - What Constitutes a Learning Problem?	View HTML	View		ECE 595ML Lecture 22.1: Is Learning Feasible? - What Constitutes a Learning Problem?
ECE 595ML Lecture 22.2: Is Learning Feasible?	View HTML	View		ECE 595ML Lecture 22.2: Is Learning Feasible?
ECE 595ML Lecture 22.3: Is Learning Feasible? - Testing vs. Training	View HTML	View		ECE 595ML Lecture 22.3: Is Learning Feasible? - Testing vs. Training
ECE 595ML Lecture 23.1: Probability Inequality - Basic Inequalities	View HTML	View		ECE 595ML Lecture 23.1: Probability Inequality - Basic Inequalities
ECE 595ML Lecture 23.2: Probability Inequality - Advanced Inequalities	View HTML	View		ECE 595ML Lecture 23.2: Probability Inequality - Advanced Inequalities
ECE 595ML Lecture 23.3: COVID-19/Probability Inequality - Hoeffding Inequality	View HTML	View		ECE 595ML Lecture 23.3: Probability Inequality - Hoeffding Inequality
ECE 595ML Lecture 24.1: Probability Approximate Correct - Two Ingredients of Generalization	View HTML	View		ECE 595ML Lecture 24.1: Probability Approximate Correct - Two Ingredients of Generalization
ECE 595ML Lecture 24.2: Probability Approximate Correct - PAC Framework	View HTML	View		ECE 595ML Lecture 24.2: Probability Approximate Correct - PAC Framework
ECE 595ML Lecture 25.1: Generalization Bound - M Hypothesis	View HTML	View		ECE 595ML Lecture 25.1: Generalization Bound - M Hypothesis
ECE 595ML Lecture 25.2: Generalization Bound	View HTML	View		ECE 595ML Lecture 25.2: Generalization Bound
ECE 595ML Lecture 25.3: Generalization Bound - Handling M Hypothesis	View HTML	View		ECE 595ML Lecture 25.3: Generalization Bound - Handling M Hypothesis
ECE 595ML Lecture 26.1: Growth Function - Overcoming the M Factor	View HTML	View		ECE 595ML Lecture 26.1: Growth Function - Overcoming the M Factor
ECE 595ML Lecture 26.2: Growth Function - Examples of mH(N)	View HTML	View		ECE 595ML Lecture 26.2: Growth Function - Examples of mH(N)
ECE 595ML Lecture 27.1: VC Dimension - From Dichotomy to Shattering	View HTML	View		ECE 595ML Lecture 27.1: VC Dimension - From Dichotomy to Shattering
ECE 595ML Lecture 27.2: VC Dimension - Example of VC Dimension	View HTML	View		ECE 595ML Lecture 27.2: VC Dimension - Example of VC Dimension
ECE 595ML Lecture 28.1: Sample and Model Complexity - Generalization Bound using VC Dimension	View HTML	View		ECE 595ML Lecture 28.1: Sample and Model Complexity - Generalization Bound using VC Dimension
ECE 595ML Lecture 28.2: Sample and Model Complexity	View HTML	View		ECE 595ML Lecture 28.2: Sample and Model Complexity
ECE 595ML Lecture 29.1: Bias and Variance - From VC Analysis to Bias-Variance	View HTML	View		ECE 595ML Lecture 29.1: Bias and Variance - From VC Analysis to Bias-Variance
ECE 595ML Lecture 29.2: Bias and Variance - Example	View HTML	View		ECE 595ML Lecture 29.2: Bias and Variance - Example
ECE 595ML Lecture 30.1: Overfit - Source of Overfit	View HTML	View		ECE 595ML Lecture 30.1: Overfit - Source of Overfit
ECE 595ML Lecture 30.2: Overfit - Analyzing Overfit	View HTML	View		ECE 595ML Lecture 30.2: Overfit - Analyzing Overfit
ECE 595ML Lecture 31.1: Regularization - Motivation for Regularization	View HTML	View		ECE 595ML Lecture 31.1: Regularization - Motivation for Regularization
ECE 595ML Lecture 31.2: Regularization - Two Regularization Techniques	View HTML	View		ECE 595ML Lecture 31.2: Regularization - Two Regularization Techniques
ECE 595ML Lecture 31.3: Regularization - Choosing a Regularization	View HTML View Flash	View		ECE 595ML Lecture 31.3: Regularization - Choosing a Regularization
ECE 595ML Lecture 32.1: Validation	View HTML	View		ECE 595ML Lecture 32.1: Validation
ECE 595ML Lecture 32.2: Validation - Model Selection	View HTML	View		ECE 595ML Lecture 32.2: Validation - Model Selection
ECE 595ML Lecture 32.3: Validation - Validation in Regularization	View HTML	View		ECE 595ML Lecture 32.3: Validation - Validation in Regularization
ECE 595ML Lecture 33.1: Adversarial Attack - An Overview	View HTML	View		ECE 595ML Lecture 33.1: Adversarial Attack - An Overview
ECE 595ML Lecture 33.2: Adversarial Attack - Basic Terminologies	View HTML	View		ECE 595ML Lecture 33.2: Adversarial Attack - Basic Terminologies