Overview of Statistical Machine Learning

Director: Nic Schraudolph (SML, NICTA and adjunct with CSL, RSISE)

The course is a general introduction to the methods and practice of statistical machine learning.

Pre-Requisites and Assumed Knowledge

A bachelor's degree in a relevant subject area; confident use of a common programming language.
Mathematical training at the 2nd year undergraduate level, including basic linear algebra and probability theory.



(all SML, NICTA and adjunct with CSL, RSISE)


NICTA on Northbourne Ave., or RSISE on the ANU campus, depending on majority of participants.



Only a pass or fail mark will be awarded. To pass the course, students must gain a pass mark on at least 3 out of at least 4 offered assignments.

Detailed Syllabus

DRAFT - subject to change at the discretion of the course organizer.
  1. Bayesian Inference Assignment 1 (theory): Ovarian Cancer Screening
    Reading: Euro coin tosses (MacKay)
  2. Maximum Likelihood Modeling Reading: Maximum Likelihood--Mixture of Gaussians (Schiele)
  3. Density Estimation Assignment 2 (programming): EM
    Reading: A Gentle Tutorial of the EM Algorithm (pages 1-3)
  4. Least Squares Regression
  5. Neural Networks Assignment 3 (programming): implement neural network
  6. Classical (Batch) Optimization Reading: Conjugate Gradient Without the Pain (chapters 1-4)
  7. Stochastic (Online) Optimization Assignment 4
  8. Overfitting, Validation, and Regularisation
  9. Reinforcement Learning (Doug Aberdeen) Assignment 5 (programming): reinforcement learning
  10. Kernel Methods 1 (Alex Smola / SVN Vishwanathan)
  11. Kernel Methods 2 (Alex Smola / SVN Vishwanathan)
    Assignment 6: kernel methods

10/05 - N. Schraudolph