Statistical learning theory 2024/25 — различия между версиями
Bauwens (обсуждение | вклад) |
Bauwens (обсуждение | вклад) |
||
| Строка 1: | Строка 1: | ||
| − | |||
== General Information == | == General Information == | ||
| − | |||
Lectures: on TBA in room TBA and in [https://us02web.zoom.us/j/82300259484?pwd=NWxXekxBeE5yMm9UTmwvLzNNNGlnUT09 zoom] by [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens] | Lectures: on TBA in room TBA and in [https://us02web.zoom.us/j/82300259484?pwd=NWxXekxBeE5yMm9UTmwvLzNNNGlnUT09 zoom] by [https://www.hse.ru/en/org/persons/160550073 Bruno Bauwens] | ||
| Строка 9: | Строка 7: | ||
To discuss the materials and practical issues, join the [https://t.me/+1begXb8SomhmODI8 telegram group] The course is similar to [http://wiki.cs.hse.ru/Statistical_learning_theory_2022 last year]. | To discuss the materials and practical issues, join the [https://t.me/+1begXb8SomhmODI8 telegram group] The course is similar to [http://wiki.cs.hse.ru/Statistical_learning_theory_2022 last year]. | ||
| + | == Homeworks == | ||
| + | |||
| + | Deadline every 2 weeks, before the lecture. The tasks are at the end of each problem list. (Problem lists will be updated, check the year.) | ||
| + | |||
| + | Before 3rd lecture: see problem lists 1 and 2. | ||
| + | Before 5th lecture: see problems lists 3 and 4. | ||
| + | Etc. | ||
| + | |||
| + | Email homeworks to brbauwens-at-gmail.com. Start the subject line with SLT-HW. Results will be here. | ||
| + | |||
| + | Late policy: 1 homework can be submitted at most 24 late without explanations. | ||
== Course materials == | == Course materials == | ||
| Строка 127: | Строка 136: | ||
Autogrades: if you only need 6/10 on the exam to have the maximal 10/10 for the course, this will be given automatically. This may happen because of extra homework questions and bonuses from quizzes. | Autogrades: if you only need 6/10 on the exam to have the maximal 10/10 for the course, this will be given automatically. This may happen because of extra homework questions and bonuses from quizzes. | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
== Colloquium == | == Colloquium == | ||
| Строка 152: | Строка 148: | ||
-- You may use handwritten notes, lecture materials from this wiki (either printed or through your PC), Mohri's book <br> | -- You may use handwritten notes, lecture materials from this wiki (either printed or through your PC), Mohri's book <br> | ||
-- You may not search on the internet or interact with other humans (e.g. by phone, forums, etc) | -- You may not search on the internet or interact with other humans (e.g. by phone, forums, etc) | ||
| − | |||
| − | |||
| − | |||
== Office hours == | == Office hours == | ||
Версия 15:55, 13 сентября 2024
Содержание
General Information
Lectures: on TBA in room TBA and in zoom by Bruno Bauwens
Seminars: on TBA in room TBA and in TBA by Nikita Lukianenko.
To discuss the materials and practical issues, join the telegram group The course is similar to last year.
Homeworks
Deadline every 2 weeks, before the lecture. The tasks are at the end of each problem list. (Problem lists will be updated, check the year.)
Before 3rd lecture: see problem lists 1 and 2. Before 5th lecture: see problems lists 3 and 4. Etc.
Email homeworks to brbauwens-at-gmail.com. Start the subject line with SLT-HW. Results will be here.
Late policy: 1 homework can be submitted at most 24 late without explanations.
Course materials
| Video | Summary | Slides | Lecture notes | Problem list | Solutions |
|---|---|---|---|---|---|
| Part 1. Online learning | |||||
| ?? Sept | Philosophy. The online mistake bound model. The halving and weighted majority algorithms. | sl01 | ch00 ch01 | prob01 | |
| ?? Sept | The perceptron algorithm. Kernels. The standard optimal algorithm. | sl02 | ch02 ch03 | prob02 | |
| ?? Sept | Prediction with expert advice. Recap probability theory (seminar). | sl03 | ch04 ch05 | prob03 | |
| Part 2. Distribution independent risk bounds | |||||
| ?? Oct | Necessity of a hypothesis class. Sample complexity in the realizable setting, examples: threshold functions and finite classes. | sl04 | ch06 | prob05 | |
| ?? Oct | Growth functions, VC-dimension and the characterization of sample comlexity with VC-dimensions | sl05 | ch07 ch08 | prob06 | |
| ?? Oct | Risk decomposition and the fundamental theorem of statistical learning theory | sl06 | ch09 | prob07 | |
| ?? Oct | Bounded differences inequality, Rademacher complexity, symmetrization, contraction lemma. | sl07 | ch10 ch11 | prob08 | |
| Part 3. Margin risk bounds with applications | |||||
| ?? Nov | Simple regression, support vector machines, margin risk bounds, and neural nets with dropout regularization | sl08 | ch12 ch13 | prob09 | |
| ?? Nov | Kernels: RKHS, representer theorem, risk bounds | sl09 | ch14 | prob10 | |
| ?? Nov | AdaBoost and the margin hypothesis | sl10 | ch15 | prob11 | |
| ?? Nov | Implicit regularization of stochastic gradient descent in overparameterized neural nets (recording with many details about the Hessian) | ch16 ch17 | |||
| ?? Dec | Part 2 of previous lecture: Hessian control and stability of the NTK. |
The lectures in October and November are based on the book:
Foundations of machine learning 2nd ed, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2018.
A gentle introduction to the materials of the first 3 lectures and an overview of probability theory, can be found in chapters 1-6 and 11-12 of the following book: Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.
Grading formula
Final grade = 0.35 * [score of homeworks] + 0.35 * [score of colloquium] + 0.3 * [score on the exam] + bonus from quizzes.
All homework questions have the same weight. Each solved extra homework task increases the score of the final exam by 1 point. At the end of the lectures there is a short quiz in which you may earn 0.1 bonus points on the final non-rounded grade.
There is no rounding except for transforming the final grade to the official grade. Arithmetic rounding is used.
Autogrades: if you only need 6/10 on the exam to have the maximal 10/10 for the course, this will be given automatically. This may happen because of extra homework questions and bonuses from quizzes.
Colloquium
Rules and questions from last year.
Date: TBA
Problems exam
TBA
-- You may use handwritten notes, lecture materials from this wiki (either printed or through your PC), Mohri's book
-- You may not search on the internet or interact with other humans (e.g. by phone, forums, etc)
Office hours
Bruno Bauwens: TBA
Nikita Lukianenko: Write in Telegram, the time is flexible
