Statistical learning theory 2018 2019 — различия между версиями

Материал из Wiki - Факультет компьютерных наук
Перейти к: навигация, поиск
м (Откат правок Seosky (обсуждение) к версии Bbauwens)
 
Строка 1: Строка 1:
−
Segeralah Mendaftar langsung di
+
 
−
* [https://macanplay.net/ Macanslot]
+
== General Information ==
−
* [https://paradox3d.net/ InaTogel]
+
 
−
* [https://macanwin.net/ QqMacan]
+
The [https://www.dropbox.com/s/8iivgt3a96yw308/syllabus_StatisticalLearning_Bach_2018_2019.pdf?dl=0 syllabus]
−
* [https://lechers.cc/ RoyalToto]
+
 
−
* [https://xenogames.net/ JayaTogelUp]
+
[https://www.dropbox.com/s/dh2n9cmhpmx97nj/colloqQuest.pdf?dl=0 Questions colloquium on 29 October.] (Lectures 1-8 updated 24/10.)
−
* [https://eeooii.info/ Raja Slot4D]
+
 
−
* [https://139.180.153.95/ Raja Slot4D]
+
Deadline homework 1: October 2nd. Questions: see seminars [https://www.dropbox.com/s/jb9mriumhtdpn8m/03sem.pdf?dl=0 3] and [https://www.dropbox.com/s/l2d9f7u77smrx4u/04sem.pdf?dl=0 4].
−
* [https://209.58.183.93/ JayaTogelUp]
+
 
−
* [https://ronin138.com/ Ronin138]
+
Deadline homework 2: October 27nd. Questions: see seminars 5-8 below.
−
* [https://addicthealious.website/ slot 4d]
+
 
−
* [https://18.142.23.191/ Macan Slot]
+
Deadline homework 3: December 11nd. Questions: see seminars 9-12 below.
−
* [https://royaltotopedia.com RoyTop88]
+
 
−
* [https://macanplay.com MacanSlot]
+
[https://www.dropbox.com/s/dy9yu1ro4k5miet/List%20of%20Students_Bruno.xlsx?dl=0  Marks]
−
* [https://macantoto88.com Toto 88]
+
 
−
* [https://jayatogelup.com.com jayatogelup]
+
Intermediate exams: October 29th.
−
[https://ronin19.com/ Ronin138]
+
 
−
[https://addicthealing.website/ slot 4d]
+
Final exam: December 20th, same system as for intermediate exams. [https://www.dropbox.com/s/uaxdredmmm5ke7t/finalTheoryQuest.pdf?dl=0 Theory questions]
−
[https://jayatogelcc.cc/ jayatogel]
+
 
−
[https://jayatogelcc.cc/ jayatogelcc]
+
Consultation: December 17th, no lecture. Students can ask questions and ask for solutions of exercises.
−
[http://northcoaststeelhead.com/ northcoaststeelhead.com]
+
 
−
[http://supremeoutlet.us/ supremeoutlet.us]
+
 
−
[http://208.78.220.231/ freebet]
+
== Course materials ==
−
[http://blackfridaymichaelkors.us/ freebet]
+
 
−
[https://supremeshirtshop.us/ supremeshirtshop.us]
+
{| class="wikitable"
−
[https://macanplayslot.web.fc2.com/ macanplay]
+
|-
−
[https://royaltotopedia4d.web.fc2.com/ royaltotopedia]
+
! Date !! Summary !! Lecture notes !! Problem list !! Solutions
−
[https://macanplay8.web.fc2.com/ macanplay slot]
+
|-
−
[https://royaltotopedia88.web.fc2.com/ royaltoto]
+
| 3 Sept || PAC-learning in the realizable setting definitions  || [https://www.dropbox.com/s/l8e8xjfe2f8tjz8/01lect.pdf?dl=0 lecture1.pdf] updated 23/09
−
[https://olxtotos.web.fc2.com/ olxtoto]
+
|| [https://www.dropbox.com/s/4ic3ce71znglmu9/01sem.pdf?dl=0 Problem list 1] || [https://www.dropbox.com/s/cixli4sghy0w01q/01solution.pdf?dl=0 Solutions 1]
−
[https://linklist.bio/OLX.TOTO olxtoto]
+
|-
−
[https://c.mi.com/thread-4131553-1-0.html olxtoto]
+
| 10 Sept || VC-dimension and growth functions || [https://www.dropbox.com/s/q1jc2dlotwdn9e2/02lect.pdf?dl=0 lecture2.pdf] updated 23/09 || [https://www.dropbox.com/s/4gimo3fij5p7lnc/02sem.pdf?dl=0 Problem list 2] || [https://www.dropbox.com/s/69pnkefexsmq6nu/02solution.pdf?dl=0 Solutions 2]
−
[https://heylink.me/FAFA138 fafa138]
+
|-
−
[https://heylink.me/FaFa138 fafa138]
+
| 17 Sept || Proof that finite VC-dimension implies PAC-learnability || [https://www.dropbox.com/s/9rfvwvf0ne95j8e/03lect.pdf?dl=0 lecture3.pdf] updated 23/09 || [https://www.dropbox.com/s/jb9mriumhtdpn8m/03sem.pdf?dl=0 Problem list 3] || [https://www.dropbox.com/s/f0gnrfxv9i7at91/03solution.pdf?dl=0 Solutions 3]
−
[https://heylink.me/Fafa138 fafa138]
+
|-
−
[https://magic.ly/FAFA138 fafa138]
+
| 24 Sept || Applications to decision trees and threshold neural networks. Agnostic PAC-learnability. || [https://www.dropbox.com/s/9oa2zg7jz2ovquf/04lect.pdf?dl=0 lecture4.pdf] || [https://www.dropbox.com/s/l2d9f7u77smrx4u/04sem.pdf?dl=0  Problem list 4] || [https://www.dropbox.com/s/t4tqtw7tdh54u2i/04solution.pdf?dl=0 Solution 4]
−
[https://fafa138slot.web.fc2.com fafa138]
+
|-
−
[https://heylink.me/FAFA138 fafa138]
+
| 1 Oct || Agnostic PAC-learnability is equivalent with finite VC-dimension, structural risk minimization || [https://www.dropbox.com/s/jsrse5qaqk2jhi1/05lect.pdf?dl=0 lecture5.pdf] 14/10 || [https://www.dropbox.com/s/etw67uq1pu5g58t/05sem.pdf?dl=0 Problem list 5] || [https://www.dropbox.com/s/6mpom53yrldcrjy/05solution.pdf?dl=0 Solution 5]
−
[http://ec2-13-250-3-146.ap-southeast-1.compute.amazonaws.com/ fafa138]
+
|-
 +
| 9 Oct || Boosting, Mohri's book pages 121-131. || [https://www.dropbox.com/s/m6tc4miryv6cs21/06lect.pdf?dl=0 lecture6.pdf] 23/10 || [https://www.dropbox.com/s/85t74k9wmibcnmr/06sem.pdf?dl=0 Problem list 6] || No solution.
 +
|-
 +
| 15 Oct || Rademacher complexity and contraction lemma (=Talagrand's lemma), Mohri's book pages 33-41 and 78-79 || [https://www.dropbox.com/s/y2vr3mrwp66cuvz/07lect.pdf?dl=0 lecture7.pdf] || [https://www.dropbox.com/s/cuo0tmfv4k2egvh/07sem.pdf?dl=0 Problem list 7] || See lecture7.pdf
 +
|-
 +
| 21 Oct || Margin theory and risk bounds for boosting. || [https://www.dropbox.com/s/o5zae3d8nw5eexw/08lect.pdf?dl=0 lecture8.pdf] || [https://www.dropbox.com/s/xg7u3ss1a0vog5j/08sem.pdf?dl=0 Problem list 8]|| See lecture6.pdf for ex. 8.6.
 +
|-
 +
| 12 Nov || Deep boosting, we study the paper [http://www.cs.nyu.edu/~mohri/pub/mboost.pdf Multi-class deep boosting], V. Kuznetsov, M Mohri, and U. Syed, Advances in Neural Information Processing Systems, p2501--2509, 2014. Notes will be provided. || [https://www.dropbox.com/s/tc7drmxwu53opzq/09lect.pdf?dl=0 lecture9.pdf] || [https://www.dropbox.com/s/lsu6tgmc767u3yd/09sem.pdf?dl=0 Problem list 9] || [https://www.dropbox.com/s/8wmswbynzx0s9hd/09sol.pdf?dl=0 Solutions 9.]
 +
|-
 +
| 19 Nov || Support vector machines, primal and dual optimization problem, risk bounds.  || See chapt. 5 of Mohri's book || [https://www.dropbox.com/s/ys37nsdfz3aa4ry/10sem.pdf?dl=0 Problem list 10]|| No solution.
 +
|-
 +
| 26 Nov || Kernels, Kernel reproducing Hilbert spaces, representer theorem, examples of kernels || [https://www.dropbox.com/s/xkic1j6r516ierl/11lect.pdf?dl=0 lecture11.pdf] || [https://www.dropbox.com/s/g3huq5aqzdaesrg/11sem.pdf?dl=0 Problem set 11] || Solutions: see lecture11.pdf
 +
|-
 +
| 3 Dec || A polynomial time improper learning algorithm for constant depth L1-regularized neural networks, from [http://www.jmlr.org/proceedings/papers/v48/zhangd16.pdf this paper].  Online algorithms: halving algorithm, weighted and exponentially weighted average algorithms. See Mohri's book Sections 7.1 and 7.2. || [https://www.dropbox.com/s/aq6798jps111l86/12lect.pdf?dl=0 lecture12.pdf] || [https://www.dropbox.com/s/o4t6smc70o1bt3t/12sem.pdf?dl=0 Problem list 12] || No solution.
 +
|-
 +
| 10 Dec || We finish online learning. Discuss the algorithm from [http://papers.nips.cc/paper/4616-bandit-algorithms-boost-brain-computer-interfaces-for-motor-task-selection-of-a-brain-controlled-button.pdf this paper].  || || See previous list. ||
 +
|}
 +
 
 +
A gentle introduction to the materials of the first 3 lectures and an overview of probability theory, can be found in chapters 1-6 and 11-12 of the following book:
 +
Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.
 +
 
 +
Afterward, we hope to cover chapters 1-8 from the book:
 +
Foundations of machine learning, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2012. These books can be downloaded from http://gen.lib.rus.ec/ .
 +
 
 +
<!--
 +
(We will study a new boosting algorithm, based on the paper: )
 +
-->
 +
 
 +
== Office hours ==
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Person !! Monday !! Tuesday !! Wednesday !! Thursday !! Friday !!
 +
|-
 +
|  Bruno Bauwens ||  16:45&ndash;19:00 || 15:05&ndash;18:00 || || ||  || Room&nbsp;620
 +
|-
 +
|}
 +
 
 +
 
 +
== Russian texts  ==
 +
 
 +
The following links might help students who have trouble with English.  A [http://www.machinelearning.ru/wiki/images/d/d9/Voron-2011-tnop.pdf  lecture] on VC-dimensions was given by K. Vorontsov.
 +
A [http://machinelearning.ru/wiki/index.php?title=%D0%A2%D0%B5%D0%BE%D1%80%D0%B8%D1%8F_%D1%81%D1%82%D0%B0%D1%82%D0%B8%D1%81%D1%82%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%BE%D0%B3%D0%BE_%D0%BE%D0%B1%D1%83%D1%87%D0%B5%D0%BD%D0%B8%D1%8F_(%D0%BA%D1%83%D1%80%D1%81_%D0%BB%D0%B5%D0%BA%D1%86%D0%B8%D0%B9%2C_%D0%9D._%D0%9A._%D0%96%D0%B8%D0%B2%D0%BE%D1%82%D0%BE%D0%B2%D1%81%D0%BA%D0%B8%D0%B9) course] on Statistical Learning Theory by Nikita Zhivotovsky is given at MIPT. Some short description about PAC learning on p136 in the [http://gen.lib.rus.ec/search.php?req=%D0%9D%D0%B0%D1%83%D0%BA%D0%B0+%D0%B8+%D0%B8%D1%81%D0%BA%D1%83%D1%81%D1%81%D1%82%D0%B2%D0%BE+%D0%BF%D0%BE%D1%81%D1%82%D1%80%D0%BE%D0%B5%D0%BD%D0%B8%D1%8F+%D0%B0%D0%BB%D0%B3%D0%BE%D1%80%D0%B8%D1%82%D0%BC%D0%BE%D0%B2%2C+%D0%BA%D0%BE%D1%82%D0%BE%D1%80%D1%8B%D0%B5+%D0%B8%D0%B7%D0%B2%D0%BB%D0%B5%D0%BA%D0%B0%D1%8E%D1%82+%D0%B7%D0%BD%D0%B0%D0%BD%D0%B8%D1%8F+%D0%B8%D0%B7+%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D1%85&lg_topic=libgen&open=0&view=simple&res=25&phrase=0&column=def book]
 +
``Наука и искусство построения алгоритмов, которые извлекают знания из данных'', Петер Флах. On [http://www.machinelearning.ru machinelearning.ru]
 +
you can find brief and clear definitions.

Текущая версия на 09:36, 26 августа 2022

General Information

The syllabus

Questions colloquium on 29 October. (Lectures 1-8 updated 24/10.)

Deadline homework 1: October 2nd. Questions: see seminars 3 and 4.

Deadline homework 2: October 27nd. Questions: see seminars 5-8 below.

Deadline homework 3: December 11nd. Questions: see seminars 9-12 below.

Marks

Intermediate exams: October 29th.

Final exam: December 20th, same system as for intermediate exams. Theory questions

Consultation: December 17th, no lecture. Students can ask questions and ask for solutions of exercises.


Course materials

Date Summary Lecture notes Problem list Solutions
3 Sept PAC-learning in the realizable setting definitions lecture1.pdf updated 23/09 Problem list 1 Solutions 1
10 Sept VC-dimension and growth functions lecture2.pdf updated 23/09 Problem list 2 Solutions 2
17 Sept Proof that finite VC-dimension implies PAC-learnability lecture3.pdf updated 23/09 Problem list 3 Solutions 3
24 Sept Applications to decision trees and threshold neural networks. Agnostic PAC-learnability. lecture4.pdf Problem list 4 Solution 4
1 Oct Agnostic PAC-learnability is equivalent with finite VC-dimension, structural risk minimization lecture5.pdf 14/10 Problem list 5 Solution 5
9 Oct Boosting, Mohri's book pages 121-131. lecture6.pdf 23/10 Problem list 6 No solution.
15 Oct Rademacher complexity and contraction lemma (=Talagrand's lemma), Mohri's book pages 33-41 and 78-79 lecture7.pdf Problem list 7 See lecture7.pdf
21 Oct Margin theory and risk bounds for boosting. lecture8.pdf Problem list 8 See lecture6.pdf for ex. 8.6.
12 Nov Deep boosting, we study the paper Multi-class deep boosting, V. Kuznetsov, M Mohri, and U. Syed, Advances in Neural Information Processing Systems, p2501--2509, 2014. Notes will be provided. lecture9.pdf Problem list 9 Solutions 9.
19 Nov Support vector machines, primal and dual optimization problem, risk bounds. See chapt. 5 of Mohri's book Problem list 10 No solution.
26 Nov Kernels, Kernel reproducing Hilbert spaces, representer theorem, examples of kernels lecture11.pdf Problem set 11 Solutions: see lecture11.pdf
3 Dec A polynomial time improper learning algorithm for constant depth L1-regularized neural networks, from this paper. Online algorithms: halving algorithm, weighted and exponentially weighted average algorithms. See Mohri's book Sections 7.1 and 7.2. lecture12.pdf Problem list 12 No solution.
10 Dec We finish online learning. Discuss the algorithm from this paper. See previous list.

A gentle introduction to the materials of the first 3 lectures and an overview of probability theory, can be found in chapters 1-6 and 11-12 of the following book: Sanjeev Kulkarni and Gilbert Harman: An Elementary Introduction to Statistical Learning Theory, 2012.

Afterward, we hope to cover chapters 1-8 from the book: Foundations of machine learning, Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalker, 2012. These books can be downloaded from http://gen.lib.rus.ec/ .


Office hours

Person Monday Tuesday Wednesday Thursday Friday
Bruno Bauwens 16:45–19:00 15:05–18:00 Room 620


Russian texts

The following links might help students who have trouble with English. A lecture on VC-dimensions was given by K. Vorontsov. A course on Statistical Learning Theory by Nikita Zhivotovsky is given at MIPT. Some short description about PAC learning on p136 in the book ``Наука и искусство построения алгоритмов, которые извлекают знания из данных, Петер Флах. On machinelearning.ru you can find brief and clear definitions.

Источник — «http://wiki.cs.hse.ru/index.php?title=Statistical_learning_theory_2018_2019&oldid=71156»