Markov Chains

Lecturers and Seminarists

Lecturer	Naumov Alexey	[anaumov@hse.ru]	T924
Seminarist	Samsonov Sergey	[svsamsonov@hse.ru]	T926

About the course

This page contains materials for Markov Chains course in 2022/2023 year, mandatory one for 1st year Master students of the MML program (HSE and Skoltech).

Grading

The final grade consists of 3 components (each is non-negative real number from 0 to 10, without any intermediate rounding) :

• O_HW for the hometasks
• O_Mid-term for the midterm exam
• O_Exam for the final exam

The formula for the final grade is

• O_Final = 0.3*O_HW + 0.3*O_Mid-term + 0.4*O_Exam + 0.1*O_Bonus

with the usual (arithmetical) rounding rule.

Table with grades

Lectures and Seminars

• Link to course materials in overleaf

Homeworks

• Homework #1, deadline: 29.01.23, 23:59
• Homework #2, deadline: 12.03.23, 23:59
• Homework #3, deadline: 30.03.23, 23:59, Collab notebook

Exam

Exam will take place on Saturday, 01.04.2023. Exam is organised at room TBD. Exam question will contain 1 theoretical question and 1 problem. Using any materials, electronic devices is allowed during preparation, but not during the answer. The proofs that were not given in the lectures/seminars can be omitted.

• List of exam questions

Midterm

Midterm will take place on Saturday, 21.01.2023. Midterm is organised at T926. Please split into 2 groups: first group starts exam at 09:00, second group starts at 11:00. You can book an exam slot below. Exam question will contain 1 theoretical question and 1 problem. Using any materials, electronic devices is allowed during preparation, but not during the answer. The proofs that were not given in the lectures/seminars can be omitted.

• List of midterm qustions
• Link to exam slots

Recommended literature (1st term)

• http://www.statslab.cam.ac.uk/~james/Markov/ - Cambridge lecture notes on discrete-time Markov Chains
• https://link.springer.com/book/10.1007%2F978-3-319-97704-1 - book by E. Moulines et al, you are mostly interested in chapters 1,2,7 and 9 (book is accessible for download through HSE network)
• https://link.springer.com/book/10.1007%2F978-3-319-62226-2 - Stochastic Calculus by P. Baldi, good overview of conditional probabilities and expectations (part 4, also accessible through HSE network)
• https://elearning.unimib.it/pluginfile.php/583708/mod_resource/content/1/1-conditional-law.pdf - Probability kernels and (regular) conditional probabilities, to the first lecture.