Icef-dse-2024-fall — различия между версиями
Bdemeshev (обсуждение | вклад) |
Bdemeshev (обсуждение | вклад) |
||
| (не показано 13 промежуточных версии этого же участника) | |||
| Строка 23: | Строка 23: | ||
* Artem Kirsanov, Key equation behind probability, [https://www.youtube.com/watch?v=KHVR587oW8I youtube]. Be careful, Artem uses notation H(P, Q) for Cross entropy (we use CE(P||Q)). | * Artem Kirsanov, Key equation behind probability, [https://www.youtube.com/watch?v=KHVR587oW8I youtube]. Be careful, Artem uses notation H(P, Q) for Cross entropy (we use CE(P||Q)). | ||
| − | *[https://exuberant-arthropod-be8.notion.site/1-02-09-5e107ea1c4054594b8f37d955db8a2b0 Конспект] аналогичной лекции на фкн на русском. | + | * [https://exuberant-arthropod-be8.notion.site/1-02-09-5e107ea1c4054594b8f37d955db8a2b0 Конспект] аналогичной лекции на фкн на русском. |
| + | |||
| + | * Keith Conrad, [https://kconrad.math.uconn.edu/blurbs/analysis/entropypost.pdf Maximal entropy] distributions. | ||
2024-09-12, lecture 2: Expected value of log-likelihood is zero. Kullback-Leibler divergence definition. Expected value calculation example. Optimizing long-run profit. Horse betting: optimal bet under private signal. | 2024-09-12, lecture 2: Expected value of log-likelihood is zero. Kullback-Leibler divergence definition. Expected value calculation example. Optimizing long-run profit. Horse betting: optimal bet under private signal. | ||
| Строка 32: | Строка 34: | ||
* Kelly, [https://www.princeton.edu/~wbialek/rome/refs/kelly_56.pdf A new interpretation of information rate]: original paper, very well written | * Kelly, [https://www.princeton.edu/~wbialek/rome/refs/kelly_56.pdf A new interpretation of information rate]: original paper, very well written | ||
| + | |||
| + | 2024-09-19, lecture 3: Horse betting: optimal bet under signal. Optimal long-term interest rate as entropy difference. How to build a tree? Entropy drop as splitting criterion. Dealing with missing values. | ||
| + | How to stop? Tree pruning. | ||
| + | |||
| + | * R2D3, Visual introduction to machine learning: [http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ decision tree] | ||
| + | |||
| + | 2024-09-26, lecture 4: Random forest | ||
| + | |||
| + | * R2D3, Visual introduction to machine learning-2: [http://www.r2d3.us/visual-intro-to-machine-learning-part-2/ bias-variance tradeoff and many trees] | ||
| + | |||
| + | 2024-10-03, lecture 5: Bootstrap: Naive bootstrap, t-stat bootstrap, bootstrap in bootstrap. | ||
| + | |||
| + | * Tim Hesterberg, [https://arxiv.org/pdf/1411.5279 What teachers should know about bootstrap?] | ||
| + | |||
| + | 2024-10-10, lecture 6: Gradient boosting for regression. Residual vector as minus gradient. Properties of logistic function. | ||
| + | |||
| + | * Alexey Natekin, Alois Knoll, [https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2013.00021/full Gradient boosting] machines. | ||
| + | |||
| + | * Cheng Li, Gentle Introduction to [https://www.chengli.io/tutorials/gradient_boosting.pdf Gradient Boosting] | ||
| + | |||
| + | 2024-10-17, lecture 7: Gradient of logit model in general form. One-to-one correspondence between probabilities and log-odds. Gradient boosting for classification. | ||
| + | |||
| + | 2024-10-24, lecture 8: Cross validation: leave-one-out, k-fold. Importance for random forest: mean decrease of impurity. Permutation based importance. | ||
| + | |||
| + | 2024-11-07, lecture 9: Differential in a matrix form, derivation of beta hat in multivariate regression. | ||
| + | |||
| + | 2024-11-07: Midterm | ||
| + | |||
| + | 2024-11-14, lecture 10: Variances and covariance in multivariate regression using matrices | ||
| + | |||
| + | 2024-11-21, lecture 11: SVD, PCA as average R2 optimization | ||
| + | |||
| + | |||
==Past courses== | ==Past courses== | ||
| − | [http://wiki.cs.hse.ru/Dse_2023-24 | + | Fall 2023: [http://wiki.cs.hse.ru/Dse_2023-24 wiki page], [https://github.com/Shuaynat/DSE-23-24/tree/main/00-exams exams]. |
| − | [http://wiki.cs.hse.ru/Icef-dse-2022-23 | + | Fall 2022: [http://wiki.cs.hse.ru/Icef-dse-2022-23 wiki]. |
Текущая версия на 00:12, 26 ноября 2024
General course info
Fall grade = 0.2 Small HAs + 0.2 Group project + 0.3 Midterm + 0.3 Final
We expect 3 practice HA and 3 theory HA.
Lecturer: Boris Demeshev
Class teachers: Yana Khassan, Shuana Pirbudagova
Lecture video recordings
Telegram group
Log Book or Tentative Plan
2024-09-05, lecture 1: Entropy, conditional entropy, joint entropy, mutual information, cross-entropy.
- Cristopher Olah, Visual Information Theory, https://colah.github.io/posts/2015-09-Visual-Information/
- Grand Sanderson, Solving Wordle using information theory, youtube.
- Artem Kirsanov, Key equation behind probability, youtube. Be careful, Artem uses notation H(P, Q) for Cross entropy (we use CE(P||Q)).
- Конспект аналогичной лекции на фкн на русском.
- Keith Conrad, Maximal entropy distributions.
2024-09-12, lecture 2: Expected value of log-likelihood is zero. Kullback-Leibler divergence definition. Expected value calculation example. Optimizing long-run profit. Horse betting: optimal bet under private signal.
- Marcin Anforowicz, Just one more paradox youtube
- Wikipedia, [Kelly Criterion https://en.wikipedia.org/wiki/Kelly_criterion]: a good article
- Kelly, A new interpretation of information rate: original paper, very well written
2024-09-19, lecture 3: Horse betting: optimal bet under signal. Optimal long-term interest rate as entropy difference. How to build a tree? Entropy drop as splitting criterion. Dealing with missing values. How to stop? Tree pruning.
- R2D3, Visual introduction to machine learning: decision tree
2024-09-26, lecture 4: Random forest
- R2D3, Visual introduction to machine learning-2: bias-variance tradeoff and many trees
2024-10-03, lecture 5: Bootstrap: Naive bootstrap, t-stat bootstrap, bootstrap in bootstrap.
- Tim Hesterberg, What teachers should know about bootstrap?
2024-10-10, lecture 6: Gradient boosting for regression. Residual vector as minus gradient. Properties of logistic function.
- Alexey Natekin, Alois Knoll, Gradient boosting machines.
- Cheng Li, Gentle Introduction to Gradient Boosting
2024-10-17, lecture 7: Gradient of logit model in general form. One-to-one correspondence between probabilities and log-odds. Gradient boosting for classification.
2024-10-24, lecture 8: Cross validation: leave-one-out, k-fold. Importance for random forest: mean decrease of impurity. Permutation based importance.
2024-11-07, lecture 9: Differential in a matrix form, derivation of beta hat in multivariate regression.
2024-11-07: Midterm
2024-11-14, lecture 10: Variances and covariance in multivariate regression using matrices
2024-11-21, lecture 11: SVD, PCA as average R2 optimization
Past courses
Fall 2022: wiki.
