Statistical learning theory 2018 2019 — различия между версиями
Bbauwens (обсуждение | вклад) |
Bbauwens (обсуждение | вклад) |
||
| Строка 29: | Строка 29: | ||
| 17 Sept || Proof that finite VC-dimension implies PAC-learnability || [https://www.dropbox.com/s/9rfvwvf0ne95j8e/03lect.pdf?dl=0 lecture3.pdf] updated 23/09 || [https://www.dropbox.com/s/jb9mriumhtdpn8m/03sem.pdf?dl=0 Problem list 3] || [https://www.dropbox.com/s/f0gnrfxv9i7at91/03solution.pdf?dl=0 Solutions 3] | | 17 Sept || Proof that finite VC-dimension implies PAC-learnability || [https://www.dropbox.com/s/9rfvwvf0ne95j8e/03lect.pdf?dl=0 lecture3.pdf] updated 23/09 || [https://www.dropbox.com/s/jb9mriumhtdpn8m/03sem.pdf?dl=0 Problem list 3] || [https://www.dropbox.com/s/f0gnrfxv9i7at91/03solution.pdf?dl=0 Solutions 3] | ||
|- | |- | ||
| − | | 24 Sept || Applications to decision trees and threshold neural networks. Agnostic PAC-learnability. || [https://www.dropbox.com/s/9oa2zg7jz2ovquf/04lect.pdf?dl=0 lecture4.pdf] || [https://www.dropbox.com/s/l2d9f7u77smrx4u/04sem.pdf?dl=0 Problem list 4] || | + | | 24 Sept || Applications to decision trees and threshold neural networks. Agnostic PAC-learnability. || [https://www.dropbox.com/s/9oa2zg7jz2ovquf/04lect.pdf?dl=0 lecture4.pdf] || [https://www.dropbox.com/s/l2d9f7u77smrx4u/04sem.pdf?dl=0 Problem list 4] || [https://www.dropbox.com/s/t4tqtw7tdh54u2i/04solution.pdf?dl=0 Solution 4] |
|- | |- | ||
| − | | 1 Oct || Agnostic PAC-learnability is equivalent with finite VC-dimension, structural risk minimization || [https://www.dropbox.com/s/jsrse5qaqk2jhi1/05lect.pdf?dl=0 lecture5.pdf] 14/10 || [https://www.dropbox.com/s/etw67uq1pu5g58t/05sem.pdf?dl=0 Problem list 5] || | + | | 1 Oct || Agnostic PAC-learnability is equivalent with finite VC-dimension, structural risk minimization || [https://www.dropbox.com/s/jsrse5qaqk2jhi1/05lect.pdf?dl=0 lecture5.pdf] 14/10 || [https://www.dropbox.com/s/etw67uq1pu5g58t/05sem.pdf?dl=0 Problem list 5] || [https://www.dropbox.com/s/6mpom53yrldcrjy/05solution.pdf?dl=0 Solution 5] |
|- | |- | ||
| − | | 9 Oct || Boosting, Mohri's book pages 121-131. || [https://www.dropbox.com/s/m6tc4miryv6cs21/06lect.pdf?dl=0 lecture6.pdf] 23/10 || [https://www.dropbox.com/s/85t74k9wmibcnmr/06sem.pdf?dl=0 Problem list 6] || | + | | 9 Oct || Boosting, Mohri's book pages 121-131. || [https://www.dropbox.com/s/m6tc4miryv6cs21/06lect.pdf?dl=0 lecture6.pdf] 23/10 || [https://www.dropbox.com/s/85t74k9wmibcnmr/06sem.pdf?dl=0 Problem list 6] || No solution. |
|- | |- | ||
| − | | 15 Oct || Rademacher complexity and contraction lemma (=Talagrand's lemma), Mohri's book pages 33-41 and 78-79 || [https://www.dropbox.com/s/y2vr3mrwp66cuvz/07lect.pdf?dl=0 lecture7.pdf] || [https://www.dropbox.com/s/cuo0tmfv4k2egvh/07sem.pdf?dl=0 Problem list 7] || | + | | 15 Oct || Rademacher complexity and contraction lemma (=Talagrand's lemma), Mohri's book pages 33-41 and 78-79 || [https://www.dropbox.com/s/y2vr3mrwp66cuvz/07lect.pdf?dl=0 lecture7.pdf] || [https://www.dropbox.com/s/cuo0tmfv4k2egvh/07sem.pdf?dl=0 Problem list 7] || Solutions in lecture7.pdf |
|- | |- | ||
| 21 Oct || Margin theory and risk bounds for boosting. || [https://www.dropbox.com/s/o5zae3d8nw5eexw/08lect.pdf?dl=0 lecture8.pdf] || [https://www.dropbox.com/s/xg7u3ss1a0vog5j/08sem.pdf?dl=0 Problem list 8]|| | | 21 Oct || Margin theory and risk bounds for boosting. || [https://www.dropbox.com/s/o5zae3d8nw5eexw/08lect.pdf?dl=0 lecture8.pdf] || [https://www.dropbox.com/s/xg7u3ss1a0vog5j/08sem.pdf?dl=0 Problem list 8]|| | ||
Версия 15:26, 24 октября 2018
General Information
The syllabus
Questions colloquium on 29 October. (Lectures 1-8 updated 24/10.)
Deadline homework 1: October 2nd. Questions: see seminars 3 and 4.
Deadline homework 2: October 27nd. Questions: see seminars 5-8 below.
Deadline homework 3: TBA.
Intermediate exams: October 29th.
Course materials
| Date | Summary | Lecture notes | Problem list | Solutions |
|---|---|---|---|---|
| 3 Sept | PAC-learning in the realizable setting definitions | lecture1.pdf updated 23/09 | Problem list 1 | Solutions 1 |
| 10 Sept | VC-dimension and growth functions | lecture2.pdf updated 23/09 | Problem list 2 | Solutions 2 |
| 17 Sept | Proof that finite VC-dimension implies PAC-learnability | lecture3.pdf updated 23/09 | Problem list 3 | Solutions 3 |
| 24 Sept | Applications to decision trees and threshold neural networks. Agnostic PAC-learnability. | lecture4.pdf | Problem list 4 | Solution 4 |
| 1 Oct | Agnostic PAC-learnability is equivalent with finite VC-dimension, structural risk minimization | lecture5.pdf 14/10 | Problem list 5 | Solution 5 |
| 9 Oct | Boosting, Mohri's book pages 121-131. | lecture6.pdf 23/10 | Problem list 6 | No solution. |
| 15 Oct | Rademacher complexity and contraction lemma (=Talagrand's lemma), Mohri's book pages 33-41 and 78-79 | lecture7.pdf | Problem list 7 | Solutions in lecture7.pdf |
| 21 Oct | Margin theory and risk bounds for boosting. | lecture8.pdf | Problem list 8 |
A gentle introduction to the materials of the first 3 lectures and an overview of probability theory, can be found in chapters 1-6 and 11-12 of the following book: Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.
Afterward, we hope to cover chapters 1-8 from the book: Foundations of machine learning, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2012. These books can be downloaded from http://gen.lib.rus.ec/ .
Office hours
| Person | Monday | Tuesday | Wednesday | Thursday | Friday | |
|---|---|---|---|---|---|---|
| Bruno Bauwens | 16:45–19:00 | 15:05–18:00 | Room 620 |
Russian texts
The following links might help students who have trouble with English. A lecture on VC-dimensions was given by K. Vorontsov. A course on Statistical Learning Theory by Nikita Zhivotovsky is given at MIPT. Some short description about PAC learning on p136 in the book ``Наука и искусство построения алгоритмов, которые извлекают знания из данных, Петер Флах. On machinelearning.ru you can find brief and clear definitions.
