Statistical learning theory 2022 — различия между версиями

Материал из Wiki - Факультет компьютерных наук
Перейти к: навигация, поиск
Строка 7: Строка 7:
 
Seminars: Friday 18h10 -- 19h30, [https://www.hse.ru/org/persons/225526439 Artur Goldman],
 
Seminars: Friday 18h10 -- 19h30, [https://www.hse.ru/org/persons/225526439 Artur Goldman],
  
−
For practical information, join the [https://t.me/+G0VKOE2-nnkwNDE0 telegram group]
+
For discussions of the materials, join the [https://t.me/+G0VKOE2-nnkwNDE0 telegram group]
  
−
The course is similar to the [http://wiki.cs.hse.ru/Statistical_learning_theory_2022 last year].
+
The course is similar to [http://wiki.cs.hse.ru/Statistical_learning_theory_2022 last year].
  
  
Строка 35: Строка 35:
 
|| ''Part 1. Online learning''  
 
|| ''Part 1. Online learning''  
 
|-
 
|-
−
| [https://drive.google.com/file/d/1WL9LSNDD1B_q6LdpfDQ8BPluNfhjWrD9/view?usp=sharing 4 Sept]  
+
| [2 Sept]  
−
|| Lecture: philosophy. Seminar: the online mistake bound model, the weighted majority, and perceptron algorithms [https://drive.google.com/drive/folders/1NXiLbhmO2Ml7jFmnLtjqhOgCoHg7yn9T?usp=sharing movies]
+
|| Lecture: philosophy. The online mistake bound model, the weighted majority, and perceptron algorithms [https://drive.google.com/drive/folders/1NXiLbhmO2Ml7jFmnLtjqhOgCoHg7yn9T?usp=sharing movies]
 
|| [https://www.dropbox.com/s/uk9awkfa827pmtf/01allSlides.pdf?dl=0 sl01]
 
|| [https://www.dropbox.com/s/uk9awkfa827pmtf/01allSlides.pdf?dl=0 sl01]
 
|| [https://www.dropbox.com/s/uvsfzb997kantoa/00book_intro.pdf?dl=0 ch00] [https://www.dropbox.com/s/6ah70h5loyrz5lx/01book_onlineMistakeBound.pdf?dl=0 ch01]
 
|| [https://www.dropbox.com/s/uvsfzb997kantoa/00book_intro.pdf?dl=0 ch00] [https://www.dropbox.com/s/6ah70h5loyrz5lx/01book_onlineMistakeBound.pdf?dl=0 ch01]
−
|| [https://www.dropbox.com/s/aoma8ma8mkd3885/01sem.pdf?dl=0 01prob (9 Sept)]
+
||  
−
|| [https://www.dropbox.com/s/sqzqlrtzr2nu8cq/01sol.pdf?dl=0 01sol]
+
||  
 
|-
 
|-
−
| [https://drive.google.com/file/d/16OoCqhh16BKQzyF-HM8RozigyJ3BBVxA/view?usp=sharing 11 Sept]
+
| [https://drive.google.com/file/d/16OoCqhh16BKQzyF-HM8RozigyJ3BBVxA/view?usp=sharing 9 Sept]
 
|| The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm.
 
|| The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm.
 
|| [https://www.dropbox.com/s/sy959ee81mov5cr/02slides.pdf?dl=0 sl02]  
 
|| [https://www.dropbox.com/s/sy959ee81mov5cr/02slides.pdf?dl=0 sl02]  
 
|| [https://www.dropbox.com/s/0029k15cbnxj2v1/02book_sequentialOptimalAlgorithm.pdf?dl=0 ch02] [https://www.dropbox.com/s/eggk7kctgox8aza/03book_perceptron.pdf?dl=0 ch03]
 
|| [https://www.dropbox.com/s/0029k15cbnxj2v1/02book_sequentialOptimalAlgorithm.pdf?dl=0 ch02] [https://www.dropbox.com/s/eggk7kctgox8aza/03book_perceptron.pdf?dl=0 ch03]
−
|| [https://www.dropbox.com/s/415nws7qi589bme/02sem.pdf?dl=0 02prob (23 Sept)]
+
||  
−
|| [https://www.dropbox.com/s/ofcctflbnxt0kx3/02sol.pdf?dl=0 02sol]
+
||  
 
|-
 
|-
−
| 18 Sept (rec to do)
+
| 16 Sept
 
|| Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory.  
 
|| Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory.  
 
|| [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 sl03]
 
|| [https://www.dropbox.com/s/a60p9b76cxusgqy/03slides.pdf?dl=0 sl03]
 
|| [https://www.dropbox.com/s/ytl6q83q6gkax3w/04book_predictionWithExperts.pdf?dl=0 ch04] [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05]
 
|| [https://www.dropbox.com/s/ytl6q83q6gkax3w/04book_predictionWithExperts.pdf?dl=0 ch04] [https://www.dropbox.com/s/l11afq1d0qn6za7/05book_introProbability.pdf?dl=0 ch05]
−
|| [https://www.dropbox.com/s/nsrcy3yxgey67lp/03sem.pdf?dl=0 03prob(30 Sept)]
+
||  
−
|| [https://www.dropbox.com/s/bg9nd01h1fhzjsi/03sol.pdf?dl=0 03sol]
+
||  
 
|-
 
|-
 
|  
 
|  
 
|| ''Part 2. Distribution independent risk bounds''  
 
|| ''Part 2. Distribution independent risk bounds''  
 
|-
 
|-
−
| [https://drive.google.com/file/d/1RHz8NgfianUQFlx8VswjiiPRvt0DoBvc/view?usp=sharing 25 Sept]
+
| [https://drive.google.com/file/d/1RHz8NgfianUQFlx8VswjiiPRvt0DoBvc/view?usp=sharing 23 Sept]
 
|| Sample complexity in the realizable setting, simple examples and bounds using VC-dimension
 
|| Sample complexity in the realizable setting, simple examples and bounds using VC-dimension
 
|| [https://www.dropbox.com/s/pi0f3wab1xna6d7/04slides.pdf?dl=0 sl04]
 
|| [https://www.dropbox.com/s/pi0f3wab1xna6d7/04slides.pdf?dl=0 sl04]
 
|| [https://www.dropbox.com/s/8xrgcugs4xv2r2p/06book_sampleComplexity.pdf?dl=0 ch06]  
 
|| [https://www.dropbox.com/s/8xrgcugs4xv2r2p/06book_sampleComplexity.pdf?dl=0 ch06]  
−
|| [https://www.dropbox.com/s/7qn2yz5fxc93rez/04sem.pdf?dl=0 04prob]
+
||  
−
|| [https://www.dropbox.com/s/xm3nhgj5d6h49nz/04sol.pdf?dl=0 04sol]
+
||  
 
|-  
 
|-  
−
| [https://drive.google.com/drive/folders/1jjyJ3eIaed64ogpR11g8M44IOikt5Mj2?usp=sharing 2 Oct]
+
| [https://drive.google.com/drive/folders/1jjyJ3eIaed64ogpR11g8M44IOikt5Mj2?usp=sharing 30 Sept]
 
|| Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions
 
|| Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions
 
|| [https://www.dropbox.com/s/rpnh6288rdb3j8m/05slides.pdf?dl=0 sl05]
 
|| [https://www.dropbox.com/s/rpnh6288rdb3j8m/05slides.pdf?dl=0 sl05]
 
|| [https://www.dropbox.com/s/ctc48w1d2vvyiyt/07book_growthFunctions.pdf?dl=0 ch07] [https://www.dropbox.com/s/jofixf9tstz0f8z/08book_VCdimension.pdf?dl=0 ch08]
 
|| [https://www.dropbox.com/s/ctc48w1d2vvyiyt/07book_growthFunctions.pdf?dl=0 ch07] [https://www.dropbox.com/s/jofixf9tstz0f8z/08book_VCdimension.pdf?dl=0 ch08]
−
|| [https://www.dropbox.com/s/zbyqxy3qp3pz79i/05sem.pdf?dl=0 05prob]
+
||  
−
|| [https://www.dropbox.com/s/a8efm18dof2zeox/05sol.pdf?dl=0 05sol]
+
||  
 
|-
 
|-
−
| [https://drive.google.com/file/d/17zynIg_CZ6cCNBig5QXmBx7VFS8peyuU/view?usp=sharing 9 Oct]
+
| [https://drive.google.com/file/d/17zynIg_CZ6cCNBig5QXmBx7VFS8peyuU/view?usp=sharing 7 Oct]
 
|| Risk decomposition and the fundamental theorem of statistical learning theory
 
|| Risk decomposition and the fundamental theorem of statistical learning theory
 
|| [https://www.dropbox.com/s/jxijka88vfanv5n/06slides.pdf?dl=0 sl06]
 
|| [https://www.dropbox.com/s/jxijka88vfanv5n/06slides.pdf?dl=0 sl06]
 
|| [https://www.dropbox.com/s/r44bwxz34qj98gg/09book_riskBounds.pdf?dl=0 ch09]
 
|| [https://www.dropbox.com/s/r44bwxz34qj98gg/09book_riskBounds.pdf?dl=0 ch09]
−
|| [https://www.dropbox.com/s/x87txc8v5v6u8vb/06sem.pdf?dl=0 06prob]
+
||  
−
|| [https://www.dropbox.com/s/ydlqu8oce3xj6ix/06sol.pdf?dl=0 06sol]
+
||  
 
|-
 
|-
−
| 16 Oct
+
| 14 Oct
 
|| Bounded differences inequality and Rademacher complexity
 
|| Bounded differences inequality and Rademacher complexity
 
|| [https://www.dropbox.com/s/kfithyq0dgcq6h8/07slides.pdf?dl=0 sl07]
 
|| [https://www.dropbox.com/s/kfithyq0dgcq6h8/07slides.pdf?dl=0 sl07]
 
|| [https://www.dropbox.com/s/5quc1jfkrvm3t71/10book_measureConcentration.pdf?dl=0 ch10] [https://www.dropbox.com/s/km0fns8n3aihauv/11book_RademacherComplexity.pdf?dl=0 ch11]
 
|| [https://www.dropbox.com/s/5quc1jfkrvm3t71/10book_measureConcentration.pdf?dl=0 ch10] [https://www.dropbox.com/s/km0fns8n3aihauv/11book_RademacherComplexity.pdf?dl=0 ch11]
−
|| [https://www.dropbox.com/s/d1rsxceqmbk5llw/07sem.pdf?dl=0 07prob]
+
||  
−
|| [https://www.dropbox.com/s/sftaa8b92ru3ii5/07sol.pdf?dl=0 07sol]
+
||  
 
|-
 
|-
 
|  
 
|  
−
|| ''Part 3. Margin risk bounds and applications''  
+
|| ''Part 3. Margin risk bounds with applications''  
 
|-
 
|-
−
| [https://drive.google.com/file/d/1L-BeDxhoHcoDrdlVTlfoMFwnWXKV46cr/view?usp=sharing 30 Oct]
+
| [https://drive.google.com/file/d/1L-BeDxhoHcoDrdlVTlfoMFwnWXKV46cr/view?usp=sharing 21 Oct]
 
|| Simple regression, support vector machines, margin risk bounds, and neural nets  
 
|| Simple regression, support vector machines, margin risk bounds, and neural nets  
 
|| [https://www.dropbox.com/s/0xrhe4732d0jshb/08slides.pdf?dl=0 sl08]
 
|| [https://www.dropbox.com/s/0xrhe4732d0jshb/08slides.pdf?dl=0 sl08]
 
|| [https://www.dropbox.com/s/cvqlwst3e69709t/12book_regression.pdf?dl=0 ch12] [https://www.dropbox.com/s/dwwxgriiaj4efn0/13book_SVM.pdf?dl=0 ch13]
 
|| [https://www.dropbox.com/s/cvqlwst3e69709t/12book_regression.pdf?dl=0 ch12] [https://www.dropbox.com/s/dwwxgriiaj4efn0/13book_SVM.pdf?dl=0 ch13]
−
|| [https://www.dropbox.com/s/qqdbrh2ll0dv03a/08sem.pdf?dl=0 08prob]
+
||  
−
|| [https://www.dropbox.com/s/9o8fyd0ff735hxu/08sol.pdf?dl=0 08sol]
+
||  
 
|-
 
|-
−
| [https://youtu.be/9FhFxLHR4eE 6 Nov]
+
| [https://youtu.be/9FhFxLHR4eE 4 Nov]
 
|| Kernels: risk bounds, RKHS, representer theorem, design
 
|| Kernels: risk bounds, RKHS, representer theorem, design
 
|| [https://www.dropbox.com/s/nhqtbekclekf6k7/09slides.pdf?dl=0 sl09]
 
|| [https://www.dropbox.com/s/nhqtbekclekf6k7/09slides.pdf?dl=0 sl09]
 
|| [https://www.dropbox.com/s/bpb9ijn2p7k19j3/14book_kernels.pdf?dl=0 ch14]
 
|| [https://www.dropbox.com/s/bpb9ijn2p7k19j3/14book_kernels.pdf?dl=0 ch14]
−
|| [https://www.dropbox.com/s/d2dmh017lw207ns/09sem.pdf?dl=0 09prob] (Nov 23)
+
||  
−
|| [https://www.dropbox.com/s/2wq9mxrqchsqujr/09sol.pdf?dl=0 09sol]
+
||  
 
|-  
 
|-  
−
| [https://youtu.be/ZBHe5RhTuzI 13 Nov]
+
| [https://youtu.be/ZBHe5RhTuzI 11 Nov]
 
|| AdaBoost and risk bounds
 
|| AdaBoost and risk bounds
 
|| [https://www.dropbox.com/s/umum3kd9439dt42/10slides.pdf?dl=0 sl10]
 
|| [https://www.dropbox.com/s/umum3kd9439dt42/10slides.pdf?dl=0 sl10]
 
|| Mohri et al, chapt 7
 
|| Mohri et al, chapt 7
−
|| [https://www.dropbox.com/s/j8s197e0mjv9qla/10sem.pdf?dl=0 10prob] (Nov 23)
+
||  
−
|| [https://www.dropbox.com/s/7lw1u8750k7s8qt/10sol.pdf?dl=0 10sol]
+
||  
 
|-
 
|-
 
|
 
|
 
|| ''Part 3. Other topics''  
 
|| ''Part 3. Other topics''  
 
|-
 
|-
−
|  
+
| 18 Nov
 +
|| Regression  I
 
||  
 
||  
−
|| Regression 
 
 
||  
 
||  
 
||  
 
||  
 
||  
 
||  
 
|-
 
|-
−
|  
+
| 25 Nov
−
||  
+
|| Regression II
 
||  
 
||  
 
||  
 
||  
Строка 128: Строка 128:
 
||
 
||
 
|-
 
|-
−
|  
+
| 2 Dec
−
||  
+
|| Multiarmed bandids I
 +
||
 +
||
 +
||
 +
||
 +
|-
 +
| 9 Dec
 +
|| Multiarmed bandids II
 
||
 
||
 
||
 
||
Строка 135: Строка 142:
 
||
 
||
 
|-
 
|-
−
| 11 Dec
+
| 16 Dec
−
|| Colloquium  
+
|| Colloquium
 
||
 
||
 
||
 
||

Версия 16:56, 30 августа 2022

General Information

Lectures: Friday 16h20 -- 17h40, Bruno Bauwens, Maxim Kaledin

Seminars: Friday 18h10 -- 19h30, Artur Goldman,

For discussions of the materials, join the telegram group

The course is similar to last year.


Homeworks

Email to brbauwens-at-gmail.com. Start the subject line with SLT-HW.

Deadline before the lecture, every other lecture.

23 Sept: see problem lists 1 and 2
07 Oct: see problem lists 3 and 4
21 Oct: see problem lists 5 and 6
4 Nov: see problem list 7
18 Nov: see problem lists 8 and 9
02 Dec: see problem lists 10 and 11


Course materials

Video Summary Slides Lecture notes Problem list Solutions
Part 1. Online learning
[2 Sept] Lecture: philosophy. The online mistake bound model, the weighted majority, and perceptron algorithms movies sl01 ch00 ch01
9 Sept The perceptron algorithm in the agnostic setting. Kernels. The standard optimal algorithm. sl02 ch02 ch03
16 Sept Prediction with expert advice and the exponentially weighted majority algorithm. Recap probability theory. sl03 ch04 ch05
Part 2. Distribution independent risk bounds
23 Sept Sample complexity in the realizable setting, simple examples and bounds using VC-dimension sl04 ch06
30 Sept Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions sl05 ch07 ch08
7 Oct Risk decomposition and the fundamental theorem of statistical learning theory sl06 ch09
14 Oct Bounded differences inequality and Rademacher complexity sl07 ch10 ch11
Part 3. Margin risk bounds with applications
21 Oct Simple regression, support vector machines, margin risk bounds, and neural nets sl08 ch12 ch13
4 Nov Kernels: risk bounds, RKHS, representer theorem, design sl09 ch14
11 Nov AdaBoost and risk bounds sl10 Mohri et al, chapt 7
Part 3. Other topics
18 Nov Regression I
25 Nov Regression II
2 Dec Multiarmed bandids I
9 Dec Multiarmed bandids II
16 Dec Colloquium


Problems exam

Dates, problems TBA

During the exam
-- You may consult notes, books and search on the internet
-- You may not interact with other humans (e.g. by phone, forums, etc)


The lectures in October and November are based on the book: Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018. This book can be downloaded from http://gen.lib.rus.ec/ .

Office hours

Person Monday Tuesday Wednesday Thursday Friday
, TBA
, TBA

It is always good to send an email in advance. Questions and feedback are welcome.


Источник — «http://wiki.cs.hse.ru/index.php?title=Statistical_learning_theory_2022&oldid=71255»