CS395 Fundamentals of Machine Learning

Inroduction to Machine Learning

• Welcome
• Scope, Terminology, Prediction, and Data
• Putting the Machine in Machine Learning
• Examples of Learning Systems
• Evaluating Learning Systems
• A Process for Building Learning Systems
• Assumptions and Reality of Learning
• About Our Setup
• The Need for Mathematical Language
• Our Software for Tackling Machine Learning
• Probability
• Linear Combinations, Weighted Sums, and Dot Products
• Notation and the Plus-One Trick
• Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity
• NumPy versus “All the Maths”
• Floating-Point Issues

Evaluation I

• Classification Tasks
• A Simple Classification Dataset
• Training and Testing: Don’t Teach to the Test
• Evaluation: Grading the Exam
• Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions
• Simple Classifier #2: Naive Bayes, Probability, and Broken Promises
• Simplistic Evaluation of Classifiers
• A Simple Regression Dataset
• Nearest-Neighbors Regression and Summary Statistics
• Linear Regression and Errors
• Optimization: Picking the Best Answer
• Simple Evaluation and Comparison of Regressors

Evaluation II

• Evaluation and Why Less Is More
• Terminology for Learning Phases
• (Re)Sampling: Making More from Less
• Break-It-Down: Deconstructing Error into Bias and Variance
• Graphical Evaluation and Comparison
• Comparing Learners with Cross-Validation
• Baseline Classifiers
• Beyond Accuracy: Metrics for Classification
• Precision-Recall Curves
• Cumulative Response and Lift Curves
• Baseline Regressors
• Additional Measures for Regression
• Residual Plots
• A First Look at Standardization

Classification and Regression Methods

• Revisiting Classification
• Decision Trees
• Support Vector Classifiers
• Logistic Regression
• Discriminant Analysis
• Assumptions, Biases, and Classifiers
• Comparison of Classifiers: Take Three
• Linear Regression in the Penalty Box: Regularization
• Piecewise Constant Regression
• Regression Trees

Clustering and Linear Regression

• Working with Text
• Clustering
• Working with Images
• Optimization
• Linear Regression from Raw Materials
• Building Logistic Regression from Raw Materials
• Neural Networks
• Probabilistic Graphical Models

Appendix A: mlwpy.py Listing