Statistical Foundations of Machine Learning

6 ECTS credits
150 h study time

Offer 2 with catalog number 4013490ENR for all students in the 2nd semester at a (E) Master - advanced level.

Semester

2nd semester

Enrollment based on exam contract

Impossible

Grading method

Grading (scale from 0 to 20)

Can retake in second session

Yes

Taught in

English

Partnership Agreement

Under agreement for exchange of courses

Faculty

Faculty of Sciences and Bioengineering Sciences

Department

Computer Science

Educational team

Pieter Libin (course titular)

Activities and contact hours

24 contact hours Lecture
24 contact hours Seminar, Exercises or Practicals
36 contact hours Independent or External Form of Study

Course Content

In this course, we Introduce the basics of Machine Learning from a statistical perspective. The focus of this course is on supervised learning, but other learning paradigms are also studied. The following topics will be addressed:

1. The Learning Problem - 2. Is Learning Feasible? - 3. The Linear Model - 4. Error and Noise - 5. Training versus Testing - 6. Theory of Generalization - 7. The Vapnik-Chervonenkis Dimension - 8. Bias-Variance Tradeoff - 9. Neural Networks - 10. Overfitting - 11. Regularization - 12. Validation - 13. Support Vector Machines - 14. Kernel Methods - 15. Bayesian learning - 16. Reinforcement learning

Additional info

Expected background knowledge: For this course, we expect the students to have a good knowledge of statistics, probability theory, calculus, and linear algebra. Familiarity with these topics is essential to understand the mathematical underpinnings of machine learning algorithms. We assume that students have experience in programming: all exercises and the exam project will be programmed in python.

Course materials: The course is based on the book by Y. Abu-Mostafa, M. Magdon-Ismail and H.-T. Lin: "Learning from Data". We use this course material: Scientific papers, course slides, book “Learning from data” and Canvas notes, lecture videos, WPO assignments and solutions (theoretical and programming exercises).

Learning Outcomes

General competencies

Introduce the basics of Machine Learning from a statistical perspective. The student has to be able to 1) understand machine learning techniques, 2) formally prove theoretical guarantees about machine learning, 3) implement these techniques in Python, 4) apply these techniques to benchmark and real-world problems, and 5) evaluate the performance of machine learning techniques.

• Knowledge and insight: After successful completion of the course the student should have insight into which problems can benefit from machine learning techniques and how to apply these techniques to the problem at hand. The student will gain insight in the studied methodologies and be able to reason about model complexities and learning guarantees.

• Use of knowledge and insight: The student should be able to apply machine learning techniques and to tune the parameters of the chosen algorithm. The use of python will enable the student to write programs to solve problems. The exercise sessions and practical exam project will challenge students to solve research questions that consider both synthetic and real-world data.

• Judgement ability: The student should be able to judge the qualities of the different machine learning techniques and their results on the problem at hand.

• Communication: The student should be able to communicate with experts about machine learning problems. The student should also be able to report and to present the results of his or her experiments to both specialists and non-specialists. The practical exam project will challenge students to collaborate with their peers and communicate their results effectively.

Grading

The final grade is composed based on the following categories:
Written Exam determines 50% of the final mark.
Practical Exam determines 50% of the final mark.

Within the Written Exam category, the following assignments need to be completed:

written exam with a relative weight of 1 which comprises 50% of the final mark.

Note: Written exam: 50% of the final grade.

Within the Practical Exam category, the following assignments need to be completed:

practical project with a relative weight of 1 which comprises 50% of the final mark.

Note: Practical project: 50% of the final grade.
For the practical project, the students get 4 assignments that they can solve in groups of two. The final exam for this project consists of giving an oral presentation of the solutions to these assignments.

Additional info regarding evaluation

Written exam: 50% of the final grade.

Practical project: 50% of the final grade.

For the practical project, the students get one assignment that entails a set of research questions, that they should solve in groups of three students. The final exam for this project consists of writing a short paper and giving an oral presentation of the solutions to these assignments.

To be eligible to take part in the written exam, the student is expected to register for the project.

Allowed unsatisfactory mark

The supplementary Teaching and Examination Regulations of your faculty stipulate whether an allowed unsatisfactory mark for this programme unit is permitted.

Academic context