Statistical learning theory 2020 — различия между версиями

Материал из Wiki - Факультет компьютерных наук
Перейти к: навигация, поиск
(не показаны 43 промежуточные версии этого же участника)
Строка 3: Строка 3:
  
 
Teachers: Bruno Bauwens and Vladimir Podolskii
 
Teachers: Bruno Bauwens and Vladimir Podolskii
 
  
 
Lectures: Saturday 9h30 - 10h50, zoom https://zoom.us/j/96210489901
 
Lectures: Saturday 9h30 - 10h50, zoom https://zoom.us/j/96210489901
  
 +
Seminars <br>
 +
- Group 1: Saturday 11h10 - 12h30, Bruno Bauwens and Vladimir Podolskii zoom https://zoom.us/j/94186131884, <br>
 +
- Group 2: Tuesday 18h, Nikita Lukyanenko, see [https://ruz.hse.ru ruz.hse.ru]
  
Seminars
+
Practical information on [https://t.me/joinchat/DZPMBRkbM5uX1l9e0pj_LQ telegram group]
  
- Group 1: Saturday 11h10 - 12h30, Bruno Bauwens and Vladimir Podolskii zoom https://zoom.us/j/94186131884,
+
Homeworks: deadlines every 2 weeks, before the lecture. <br>
 +
- Sat 3 Oct: see problem lists 1 and 2 <br>
 +
- Sat 17 Oct: see problem lists 3 and 4<br>
 +
- Sat 31 Oct: see problem list 5<br>
 +
- Sat 14 Nov: see problem lists 6 and 7<br>
 +
- Sat 28 Nov: see problem lists 8 and 9<br>
 +
- Sat 12 Dec: see problem lists 10 and 11
  
- Group 2: Tuesday 18h, Nikita Lukyanenko, see [https://ruz.hse.ru ruz.hse.ru]
+
[https://www.dropbox.com/s/z5q5ib34abybj06/scores.ods?dl=0 Results.]
  
 
+
Email homeworks to Brbauwens <at> gmail.com. Start the subject line with SLT-HW. You may submit handwritten solutions.  
Homeworks: deadlines every 2 weeks, <span style="color:red">Sat 26 Sept</span>, 10 Oct, etc. before the lecture. [https://www.dropbox.com/s/z5q5ib34abybj06/scores.ods?dl=0 Results.]
+
  
 
== Course materials ==
 
== Course materials ==
Строка 27: Строка 34:
 
|| [https://www.dropbox.com/s/pehka8xyu5hlpis/slides01.pdf?dl=0 slides1.pdf]
 
|| [https://www.dropbox.com/s/pehka8xyu5hlpis/slides01.pdf?dl=0 slides1.pdf]
 
||  
 
||  
|| [https://www.dropbox.com/s/fbdew1vdzskenie/01sem.pdf?dl=0 Problem list 1] <span style="color:red">Update 16.09, prob 1.2</span>
+
|| [https://www.dropbox.com/s/fbdew1vdzskenie/01sem.pdf?dl=0 Problem list 1] <span style="color:red">Update 26.09, prob 1.7</span>
 
|| [https://www.dropbox.com/s/rn8nv9y0db61a0h/01sol.pdf?dl=0 Solutions 1]
 
|| [https://www.dropbox.com/s/rn8nv9y0db61a0h/01sol.pdf?dl=0 Solutions 1]
 
|-
 
|-
Строка 37: Строка 44:
 
|| [https://www.dropbox.com/s/0g5gw3yrjjjzz07/02sol.pdf?dl=0 Solutions 2]
 
|| [https://www.dropbox.com/s/0g5gw3yrjjjzz07/02sol.pdf?dl=0 Solutions 2]
 
|-
 
|-
| 26 Sept || Risk bounds and the fundamental theorem of statistical learning theory || || || || ||
+
| 26 Sept || Risk bounds and the fundamental theorem of statistical learning theory  
 +
|| [https://www.dropbox.com/s/njekia6g8t0x5mb/03lect.pdf?dl=0 lecture3.pdf]
 +
|| [https://www.dropbox.com/s/at4eph4mv9gfnp1/03slides.pdf?dl=0 slides3.pdf]
 +
||  
 +
|| [https://www.dropbox.com/s/nvb25e0ccebbz2a/03sem.pdf?dl=0 Problem list 3]
 +
|| [https://www.dropbox.com/s/5jbl0xul25mrbg1/03sol.pdf?dl=0 Solutions 3]
 
|-
 
|-
| 03 Nov || Rademacher complexity and margin assumption || || || || ||
+
| 03 Oct || Rademacher complexity  
 +
|| [https://www.dropbox.com/s/ggw79gau85a4mcl/04lect.pdf?dl=0 lecture4.pdf]
 +
|| [https://www.dropbox.com/s/pd2ockzxqdfo66t/04slides.pdf?dl=0 slides4.pdf]
 +
||  
 +
|| [https://www.dropbox.com/s/rbx6jwlusnwhkzn/04sem.pdf?dl=0 Problem list 4] <span style="color:red">Update 09.10, prob 4.3,4.6</span>
 +
|| [https://www.dropbox.com/s/nhxkxfjajzsgfnf/04sol.pdf?dl=0 Solutions 4]
 
|-
 
|-
| ... || || || || || ||
+
| 10 Oct || Support vector machines and risk bounds
 +
|| Chapt 5, Mohri et al, see below
 +
|| [https://www.dropbox.com/s/q2onm9o6wgceg5i/05slides.pdf?dl=0 slides5.pdf]
 +
||
 +
|| [https://www.dropbox.com/s/upv70of97fqpx5f/05sem.pdf?dl=0 Problem list 5] <span style="color:red">Update 12.10, added background info</span>
 +
|| [https://www.dropbox.com/s/jfneptto1qoug1g/05sol.pdf?dl=0 Solutions 5]
 +
|-
 +
| 17 Oct || Support vector machines and kernels.
 +
|| [https://www.dropbox.com/s/7zi67710l4zdnyv/06lect.pdf?dl=0 lecture6.pdf]
 +
|| [https://www.dropbox.com/s/tot9akaoonja1zp/06slides.pdf?dl=0 slides6.pdf]
 +
||
 +
|| [https://www.dropbox.com/s/y7w3srgsrp9d7m0/06sem.pdf?dl=0 Problem list 6]
 +
||
 +
|-
 +
| 31 Oct || Adaboost
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 07 Nov || Online learning 1
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 14 Nov || Online learning 2
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 21 Nov || Active learning
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 28 Nov || Unsupervised learning
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 5 Dec || Optional: Neural networks and stochastic gradient descent
 +
||
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 12 Dec || Optional: Neural networks and stochastic gradient descent
 +
||  
 +
||  
 +
||  
 +
||  
 +
||
 
|}
 
|}
  
Строка 48: Строка 126:
 
Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.-->
 
Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.-->
  
<!-- Afterward, we hope to cover chapters 1-8 from the book:
+
The lectures in October are based on the book:
Foundations of machine learning, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2012. These books can be downloaded from http://gen.lib.rus.ec/ .
+
Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .
  
(We will study a new boosting algorithm, based on the paper: )
+
In November, we follow the [http://machinelearning.ru/wiki/index.php?title=%D0%A2%D0%B5%D0%BE%D1%80%D0%B8%D1%8F_%D1%81%D1%82%D0%B0%D1%82%D0%B8%D1%81%D1%82%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%BE%D0%B3%D0%BE_%D0%BE%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D1%8F_(%D0%BA%D1%83%D1%80%D1%81_%D0%BB%D0%B5%D0%BA%D1%86%D0%B8%D0%B9%2C_%D0%9D._%D0%9A._%D0%96%D0%B8%D0%B2%D0%BE%D1%82%D0%BE%D0%B2%D1%81%D0%BA%D0%B8%D0%B9) lecture notes] by Н. К. Животовский
-->
+
  
 
== Office hours ==
 
== Office hours ==

Версия 00:34, 17 октября 2020

General Information

Teachers: Bruno Bauwens and Vladimir Podolskii

Lectures: Saturday 9h30 - 10h50, zoom https://zoom.us/j/96210489901

Seminars
- Group 1: Saturday 11h10 - 12h30, Bruno Bauwens and Vladimir Podolskii zoom https://zoom.us/j/94186131884,
- Group 2: Tuesday 18h, Nikita Lukyanenko, see ruz.hse.ru

Practical information on telegram group

Homeworks: deadlines every 2 weeks, before the lecture.
- Sat 3 Oct: see problem lists 1 and 2
- Sat 17 Oct: see problem lists 3 and 4
- Sat 31 Oct: see problem list 5
- Sat 14 Nov: see problem lists 6 and 7
- Sat 28 Nov: see problem lists 8 and 9
- Sat 12 Dec: see problem lists 10 and 11

Results.

Email homeworks to Brbauwens <at> gmail.com. Start the subject line with SLT-HW. You may submit handwritten solutions.

Course materials

Date Summary Lecture notes Slides Video Problem list Solutions
12 Sept Introduction and sample complexity in the realizable setting lecture1.pdf slides1.pdf Problem list 1 Update 26.09, prob 1.7 Solutions 1
19 Sept VC-dimension and sample complexity lecture2.pdf slides2.pdf Chapt 2,3 Problem list 2 Solutions 2
26 Sept Risk bounds and the fundamental theorem of statistical learning theory lecture3.pdf slides3.pdf Problem list 3 Solutions 3
03 Oct Rademacher complexity lecture4.pdf slides4.pdf Problem list 4 Update 09.10, prob 4.3,4.6 Solutions 4
10 Oct Support vector machines and risk bounds Chapt 5, Mohri et al, see below slides5.pdf Problem list 5 Update 12.10, added background info Solutions 5
17 Oct Support vector machines and kernels. lecture6.pdf slides6.pdf Problem list 6
31 Oct Adaboost
07 Nov Online learning 1
14 Nov Online learning 2
21 Nov Active learning
28 Nov Unsupervised learning
5 Dec Optional: Neural networks and stochastic gradient descent
12 Dec Optional: Neural networks and stochastic gradient descent


The lectures in October are based on the book: Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .

In November, we follow the lecture notes by Н. К. Животовский

Office hours

Person Monday Tuesday Wednesday Thursday Friday
Bruno Bauwens 14h-18h 16h15-20h Room S834 Pokrovkaya 11

It is always good to send an email in advance. Questions are welcome.