Statistical learning theory 2021 — различия между версиями
Bbauwens (обсуждение | вклад) |
Bbauwens (обсуждение | вклад) |
||
| Строка 54: | Строка 54: | ||
|| Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. | || Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. | ||
|| [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 03sl] | || [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 03sl] | ||
| − | || [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05] | + | || [https://www.dropbox.com/s/ytl6q83q6gkax3w/04book_predictionWithExperts.pdf?dl=0 ch04] [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05] |
|| [https://www.dropbox.com/s/nsrcy3yxgey67lp/03sem.pdf?dl=0 03prob(30 Sept)] | || [https://www.dropbox.com/s/nsrcy3yxgey67lp/03sem.pdf?dl=0 03prob(30 Sept)] | ||
|| [https://www.dropbox.com/s/bg9nd01h1fhzjsi/03sol.pdf?dl=0 03sol] | || [https://www.dropbox.com/s/bg9nd01h1fhzjsi/03sol.pdf?dl=0 03sol] | ||
Версия 01:37, 6 октября 2021
General Information
Teachers: Bruno Bauwens and Nikita Lukianenko
Lectures: Saturday 14:40 - 16:00. The lectures are Pokrovkaya and also streamed here in zoom.
Seminars: Tuesday 16:20 - 17:40. The seminars are Pokrovkaya and also streamed here in zoom.
See ruz for the rooms.
Practical information on a telegram group. Join here.
The course is similar last year, except for the order of topics and part 3.
Homeworks
Email to brbauwens-at-gmail.com. Start the subject line with SLT-HW.
Deadline before the lecture, every 2 weeks.
25 Sept: see problem lists 1 and 2
09 Oct: see problem lists 3 Update 30/9 and 4
Etc.
Course materials
| Video | Summary | Slides | Lecture notes | Problem list | Solutions |
|---|---|---|---|---|---|
| Part 1. Online learning | |||||
| 4 Sept | Lecture: philosophy. Seminar: the online mistake bound model, the weighted majority, and perceptron algorithms movies | 01sl | 00ch 01ch | 01prob (9 Sept) | 01sol |
| 11 Sept | The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm. | 02sl | 02ch 03ch | 02prob (23 Sept) | 02sol |
| 18 Sept (rec to do) | Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. | 03sl | ch04 ch05 | 03prob(30 Sept) | 03sol |
| Part 2. Risk bounds for binary classification | |||||
| 25 Sept | Sample complexity in the realizable setting, simple examples and bounds using VC-dimension | sl04 | ch06 | 04prob | 04sol |
| 2 Oct | Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions | sl05 | ch07 ch08 | 05prob | |
| 9 Oct | Risk decomposition and the fundamental theorem of statistical learning theory | ||||
| 16 Oct | Rademacher complexity | ||||
| 29 Oct | Support vector machines and margin risk bounds | ||||
| 6 Nov | Kernels: risk bounds, design, and representer theorem | ||||
| 13 Nov | AdaBoost and risk bounds | ||||
| Part 3. Other topics | |||||
| 20 Nov | Clustering | ||||
| 27 Nov | Dimensionality reduction and the Johnson-Lindenstrauss lemma | ||||
| 4 Dec | Active learning | ||||
| 11 Dec | Colloquium |
The lectures in October and November are based on the book: Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .
Office hours
| Person | Monday | Tuesday | Wednesday | Thursday | Friday | |
|---|---|---|---|---|---|---|
| Bruno Bauwens, Zoom | 12h30-14h30 | 14h-20h | Room S834 Pokrovkaya 11 | |||
| Nikita Lukianenko, Telegram | 14h30-16h30 | 14h30-16h30 | Room S831 Pokrovkaya 11 |
It is always good to send an email in advance. Questions and feedback are welcome.
I am traveling from Sept 12 -- Sept 30 and Oct 16 -- Oct 26. On Fridays I'm available till 16h30.
