# Machine Learning I: Algorithms and Applications

Week Exercise Lecture Mon. Thu. Content Content 1 (no exercise) 24.10. - organisational matters - Machine Learning:    Introduction and History (K/S) Homework: Perceptron learning rule 2 28.10. - homework review - Tutorial (M):    probability theory 31.10. Bayes Rule (S): - central role in statistics - derivation and formula - use for machine inference Homework: Ovarian cancer screening 3 04.11. - homework review - Tutorial (M):    differentiation 07.11. ML Modeling (S): - modeling tasks: regression, classi-   fication, density estimation - maximum likelihood (ML) loss fn.s Homework: See page 12 of lecture notes Reading:  Maximum Likelihood - Mixture of Gaussians 4 11.11. - homework review - lecture review 14.11. Density Estimation (S): - parametric vs. non-parametric - classification via density estim. - semi-parametric & mixture models - Expectation-Maximisation (EM) Reading: pages 1-3 of  A Gentle Tutorial of the EM Algorithm Reading: chapters 1-4 of Conjugate Gradient Without the Pain 5 18.11. - lecture review 21.11. Least-Squares Regression (S): - linear vs. non-linear models - simple gradient descent, SVD - basis functions, generalized LS - classification via regression Homework: questions 6 25.11. - homework review - lecture review 28.11. Overfitting & Validation (M): - problem of overfitting - empirical vs. true risk - cross-validation Homework: questions 7 02.12. - homework review - lecture review 05.12. Penalization & Model Selection (M): - Penalization - Ockham's razor - structural risk minimization - minimum description length Homework: questions 8 09.12. lecture review 12.12. Neural Networks (M): - biological background - learning in neural networks - backpropagation algorithm Reading: Lectures 1 and 2 of NN Course 9 16.12. review (M): backpropagation 19.12. Training Methods (M): - learning rate adaptation - quasi-Newton methods - conjugate gradient Programming Assignment: Handout 10 (no exercise) 09.01. Classification (K): - Fisher's linear discriminants - k-nearest neighbor - vector quantisation 11 13.01. lecture review 16.01. TBA 12 20.01. lecture review 23.01. Dimensionality Reduction (K): - curse of dimensionality - principal components analysis - nonlinear autoencoding 13 27.01. lecture review 30.01. Self-Organising Maps (K) 14 03.02. lecture review 06.02. Summary lecture