Statistical learning theory 2021 — различия между версиями
Bbauwens (обсуждение | вклад) |
|||
(не показано 39 промежуточных версии 2 участников) | |||
Строка 8: | Строка 8: | ||
Lectures: Saturday 14:40 - 16:00. The lectures are Pokrovkaya and also streamed [https://us02web.zoom.us/j/82173400975?pwd=L1lhTzFTc2lGem5BVFdRcFEyVUhqZz09 here] in zoom. | Lectures: Saturday 14:40 - 16:00. The lectures are Pokrovkaya and also streamed [https://us02web.zoom.us/j/82173400975?pwd=L1lhTzFTc2lGem5BVFdRcFEyVUhqZz09 here] in zoom. | ||
− | + | Seminars: Tuesday 16:20 - 17:40. The seminars are Pokrovkaya and also streamed [https://meet.google.com/ber-yzns-hxz here] in google.meet. | |
− | + | ||
− | Seminars: Tuesday 16:20 - 17:40. The seminars are Pokrovkaya and also streamed [https:// | + | |
See [https://ruz.hse.ru/ruz/main ruz] for the rooms. | See [https://ruz.hse.ru/ruz/main ruz] for the rooms. | ||
− | Practical information on [https://t.me/joinchat/IER2-8hc0wUxNDQ0 | + | Practical information on a telegram group. [https://t.me/joinchat/IER2-8hc0wUxNDQ0 Join here.] |
The course is similar [http://wiki.cs.hse.ru/Statistical_learning_theory_2020 last year], except for the order of topics and part 3. | The course is similar [http://wiki.cs.hse.ru/Statistical_learning_theory_2020 last year], except for the order of topics and part 3. | ||
Строка 20: | Строка 18: | ||
== Homeworks == | == Homeworks == | ||
− | Email to brbauwens-at-gmail.com. | + | Email to brbauwens-at-gmail.com. Start the subject line with SLT-HW. [https://www.dropbox.com/s/taskzhu0nj5motd/scores.ods?dl=0 Results] |
− | Deadline before the lecture, every | + | Deadline before the lecture, every other lecture. |
− | + | 25 Sept: see problem lists 1 and 2 | |
− | 09 Oct: see problem lists 3 and 4 | + | 09 Oct: see problem lists 3 <span style="color:red">Update 30/9</span> and 4 |
Etc. | Etc. | ||
Строка 37: | Строка 35: | ||
|- | |- | ||
| | | | ||
− | || ''Part 1. Online learning'' | + | || ''Part 1. Online learning'' |
|- | |- | ||
| [https://drive.google.com/file/d/1WL9LSNDD1B_q6LdpfDQ8BPluNfhjWrD9/view?usp=sharing 4 Sept] | | [https://drive.google.com/file/d/1WL9LSNDD1B_q6LdpfDQ8BPluNfhjWrD9/view?usp=sharing 4 Sept] | ||
Строка 50: | Строка 48: | ||
|| [https://www.dropbox.com/s/sy959ee81mov5cr/02slides.pdf?dl=0 02sl] | || [https://www.dropbox.com/s/sy959ee81mov5cr/02slides.pdf?dl=0 02sl] | ||
|| [https://www.dropbox.com/s/0029k15cbnxj2v1/02book_sequentialOptimalAlgorithm.pdf?dl=0 02ch] [https://www.dropbox.com/s/eggk7kctgox8aza/03book_perceptron.pdf?dl=0 03ch] | || [https://www.dropbox.com/s/0029k15cbnxj2v1/02book_sequentialOptimalAlgorithm.pdf?dl=0 02ch] [https://www.dropbox.com/s/eggk7kctgox8aza/03book_perceptron.pdf?dl=0 03ch] | ||
− | || [https://www.dropbox.com/s/415nws7qi589bme/02sem.pdf?dl=0 02prob ( | + | || [https://www.dropbox.com/s/415nws7qi589bme/02sem.pdf?dl=0 02prob (23 Sept)] |
|| [https://www.dropbox.com/s/ofcctflbnxt0kx3/02sol.pdf?dl=0 02sol] | || [https://www.dropbox.com/s/ofcctflbnxt0kx3/02sol.pdf?dl=0 02sol] | ||
|- | |- | ||
− | | 18 Sept ( | + | | 18 Sept (rec to do) |
− | || Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory | + | || Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. |
− | || | + | || [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 03sl] |
− | || | + | || [https://www.dropbox.com/s/ytl6q83q6gkax3w/04book_predictionWithExperts.pdf?dl=0 ch04] [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05] |
− | || | + | || [https://www.dropbox.com/s/nsrcy3yxgey67lp/03sem.pdf?dl=0 03prob(30 Sept)] |
− | || | + | || [https://www.dropbox.com/s/bg9nd01h1fhzjsi/03sol.pdf?dl=0 03sol] |
|- | |- | ||
| | | | ||
− | || ''Part 2. | + | || ''Part 2. Risk bounds for binary classification'' |
|- | |- | ||
− | | 25 Sept | + | | [https://drive.google.com/file/d/1RHz8NgfianUQFlx8VswjiiPRvt0DoBvc/view?usp=sharing 25 Sept] |
− | || Sample complexity in the realizable setting, simple | + | || Sample complexity in the realizable setting, simple examples and bounds using VC-dimension |
− | || | + | || [https://www.dropbox.com/s/pi0f3wab1xna6d7/04slides.pdf?dl=0 sl04] |
− | || | + | || [https://www.dropbox.com/s/8xrgcugs4xv2r2p/06book_sampleComplexity.pdf?dl=0 ch06] |
− | || | + | || [https://www.dropbox.com/s/7qn2yz5fxc93rez/04sem.pdf?dl=0 04prob] |
− | || | + | || [https://www.dropbox.com/s/xm3nhgj5d6h49nz/04sol.pdf?dl=0 04sol] |
|- | |- | ||
| 2 Oct | | 2 Oct | ||
+ | || Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions | ||
+ | || [https://www.dropbox.com/s/rpnh6288rdb3j8m/05slides.pdf?dl=0 sl05] | ||
+ | || [https://www.dropbox.com/s/ctc48w1d2vvyiyt/07book_growthFunctions.pdf?dl=0 ch07] [https://www.dropbox.com/s/jofixf9tstz0f8z/08book_VCdimension.pdf?dl=0 ch08] | ||
+ | || [https://www.dropbox.com/s/zbyqxy3qp3pz79i/05sem.pdf?dl=0 05prob] | ||
+ | || [https://www.dropbox.com/s/a8efm18dof2zeox/05sol.pdf?dl=0 05sol] | ||
+ | |- | ||
+ | | [https://drive.google.com/file/d/17zynIg_CZ6cCNBig5QXmBx7VFS8peyuU/view?usp=sharing 9 Oct] | ||
|| Risk decomposition and the fundamental theorem of statistical learning theory | || Risk decomposition and the fundamental theorem of statistical learning theory | ||
− | || | + | || [https://www.dropbox.com/s/jxijka88vfanv5n/06slides.pdf?dl=0 sl06] |
− | || | + | || [https://www.dropbox.com/s/r44bwxz34qj98gg/09book_riskBounds.pdf?dl=0 ch09] |
− | || | + | || [https://www.dropbox.com/s/x87txc8v5v6u8vb/06sem.pdf?dl=0 06prob] |
− | || | + | || [https://www.dropbox.com/s/ydlqu8oce3xj6ix/06sol.pdf?dl=0 06sol] |
|- | |- | ||
− | | | + | | 16 Oct |
− | || Rademacher complexity | + | || Bounded differences inequality and Rademacher complexity |
− | || | + | || [https://www.dropbox.com/s/kfithyq0dgcq6h8/07slides.pdf?dl=0 sl07] |
− | || | + | || [https://www.dropbox.com/s/5quc1jfkrvm3t71/10book_measureConcentration.pdf?dl=0 ch10] [https://www.dropbox.com/s/km0fns8n3aihauv/11book_RademacherComplexity.pdf?dl=0 ch11] |
|| | || | ||
|| | || | ||
|- | |- | ||
− | | | + | | 29 Oct |
|| Support vector machines and margin risk bounds | || Support vector machines and margin risk bounds | ||
|| | || | ||
Строка 91: | Строка 96: | ||
|| | || | ||
|- | |- | ||
− | | | + | | 6 Nov |
|| Kernels: risk bounds, design, and representer theorem | || Kernels: risk bounds, design, and representer theorem | ||
|| | || | ||
Строка 98: | Строка 103: | ||
|| | || | ||
|- | |- | ||
− | | | + | | 13 Nov |
|| AdaBoost and risk bounds | || AdaBoost and risk bounds | ||
|| | || | ||
Строка 106: | Строка 111: | ||
|- | |- | ||
| | | | ||
− | || ''Part 3. Other topics'' | + | || ''Part 3. Other topics'' |
|- | |- | ||
− | | | + | | 20 Nov |
|| Clustering | || Clustering | ||
|| | || | ||
Строка 115: | Строка 120: | ||
|| | || | ||
|- | |- | ||
− | | | + | | 27 Nov |
|| Dimensionality reduction and the Johnson-Lindenstrauss lemma | || Dimensionality reduction and the Johnson-Lindenstrauss lemma | ||
|| | || | ||
Строка 122: | Строка 127: | ||
|| | || | ||
|- | |- | ||
− | | | + | | 4 Dec |
|| Active learning | || Active learning | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
|| | || | ||
|| | || | ||
Строка 244: | Строка 242: | ||
|- | |- | ||
| [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens], [https://zoom.us/j/5579743402 Zoom] || || 12h30-14h30 || || || 14h-20h || Room S834 Pokrovkaya 11 | | [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens], [https://zoom.us/j/5579743402 Zoom] || || 12h30-14h30 || || || 14h-20h || Room S834 Pokrovkaya 11 | ||
+ | |- | ||
+ | | [https://www.hse.ru/org/persons/225553845 Nikita Lukianenko], [https://t.me/vaulty Telegram] || || 14h30-16h30 || 14h30-16h30 || || || Room S831 Pokrovkaya 11 | ||
|- | |- | ||
|} | |} |
Версия 01:13, 25 октября 2021
General Information
Teachers: Bruno Bauwens and Nikita Lukianenko
Lectures: Saturday 14:40 - 16:00. The lectures are Pokrovkaya and also streamed here in zoom.
Seminars: Tuesday 16:20 - 17:40. The seminars are Pokrovkaya and also streamed here in google.meet.
See ruz for the rooms.
Practical information on a telegram group. Join here.
The course is similar last year, except for the order of topics and part 3.
Homeworks
Email to brbauwens-at-gmail.com. Start the subject line with SLT-HW. Results
Deadline before the lecture, every other lecture.
25 Sept: see problem lists 1 and 2
09 Oct: see problem lists 3 Update 30/9 and 4
Etc.
Course materials
Video | Summary | Slides | Lecture notes | Problem list | Solutions |
---|---|---|---|---|---|
Part 1. Online learning | |||||
4 Sept | Lecture: philosophy. Seminar: the online mistake bound model, the weighted majority, and perceptron algorithms movies | 01sl | 00ch 01ch | 01prob (9 Sept) | 01sol |
11 Sept | The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm. | 02sl | 02ch 03ch | 02prob (23 Sept) | 02sol |
18 Sept (rec to do) | Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. | 03sl | ch04 ch05 | 03prob(30 Sept) | 03sol |
Part 2. Risk bounds for binary classification | |||||
25 Sept | Sample complexity in the realizable setting, simple examples and bounds using VC-dimension | sl04 | ch06 | 04prob | 04sol |
2 Oct | Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions | sl05 | ch07 ch08 | 05prob | 05sol |
9 Oct | Risk decomposition and the fundamental theorem of statistical learning theory | sl06 | ch09 | 06prob | 06sol |
16 Oct | Bounded differences inequality and Rademacher complexity | sl07 | ch10 ch11 | ||
29 Oct | Support vector machines and margin risk bounds | ||||
6 Nov | Kernels: risk bounds, design, and representer theorem | ||||
13 Nov | AdaBoost and risk bounds | ||||
Part 3. Other topics | |||||
20 Nov | Clustering | ||||
27 Nov | Dimensionality reduction and the Johnson-Lindenstrauss lemma | ||||
4 Dec | Active learning | ||||
11 Dec | Colloquium |
The lectures in October and November are based on the book: Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .
Office hours
Person | Monday | Tuesday | Wednesday | Thursday | Friday | |
---|---|---|---|---|---|---|
Bruno Bauwens, Zoom | 12h30-14h30 | 14h-20h | Room S834 Pokrovkaya 11 | |||
Nikita Lukianenko, Telegram | 14h30-16h30 | 14h30-16h30 | Room S831 Pokrovkaya 11 |
It is always good to send an email in advance. Questions and feedback are welcome.
I am traveling from Sept 12 -- Sept 30 and Oct 16 -- Oct 26. On Fridays I'm available till 16h30.