RL 2023

Lecturers and Seminarists

Lecturer	Alexey Naumov	[anaumov@hse.ru]	T924
Seminarist	Ilya Levin	[tg: @levensons]	T926

About the course

This page contains materials for Mathematical Foundations of Reinforcement learning course in 2023 year, optional one for 2nd year Master students of the Math of Machine Learning program (HSE and Skoltech).

Grading

The final grade consists of 2 components (each is non-negative real number from 0 to 10, without any intermediate rounding) :

• O_HW for the hometasks
• O_Exam for the exam

The formula for the final grade is

• O_Final = 0.6*O_HW + 0.4*O_Exam

with the usual (arithmetical) rounding rule.

Course materials

• Lectures and seminars notes
• Notebook for the first seminar

Homeworks

• HW 1

Recommended literature

• Sebastien Bubek, Nicolo Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Chapter 2. http://sbubeck.com/SurveyBCB12.pdf
• Richard S. Sutton, Andrew G. Barto. Reinforcement Learning: An Introduction. Chapter 2. http://incompleteideas.net/book/the-book-2nd.html;
• Botao Hao et al. Bootstrapping Upper Confidence Bound. https://arxiv.org/abs/1906.05247
• Aleksandrs Slivkins. Introduction to Multi-Armed Bandits. https://arxiv.org/abs/1904.07272 [Chapter 1]