Random processes

Course guarantor:

Lecturer:

Tutor:

Supervisor:

Department of Applied Mathematics

Synopsis:

Random processes, simulation of random processes, Markov chains, stationary distribution, queuing models, hidden Markov models.

Requirements:

The basic knowledge of probability theory is expected, at least in extend of BI(E)-PST at FIT.

Syllabus of lectures:

1. Random process, trajectories and distribution of random processes.

2. Markov condition, Chapman-Kolmogorov equation, stationary distribution.

3. State classification, limit behaviour, absorption probabilities.

4. Markov chain Monte-Carlo methods.

5. Poisson process, definition, non-homogeneous Poisson process.

6. Markov chain with continuous time, intenzity matrix, Kolmogorov equations.

7. Timing of Markov chain by Poisson process, trajectories.

8. Queuing process, Kendall notation, simulations.

9. Distribution of queuing models, relation to Markov chains with continuous time, Little theorem.

10. State processes, hidden Markov chains.

Syllabus of tutorials:

1. Trajectories and distribution of random process.

2. Markov chain with discrete time, stationary distribution.

3. State classification.

4. Markov chain Monte-Carlo methods.

5. Poisson process.

6. Markov chain with continuous time.

7. Timing of Markov chain by Poisson process.

8. Queuing process.

9. Little theorem.

10. State processes, hidden Markov chains.

Study Objective:

The aim of the course is to familiarize students with the basic principles of random processes. The course focuses on both the mathematical description of the process and methods of simulation and application of such processes. The course focuses mainly on Markov chains with a countable set of states, both with continuous and discrete time. The course aims to apply these processes in more general models, such as queuing models or hidden Markov models.

Study materials:

1. R. Durrett: Essentials of stochastic processes, 2nd Edition. New York: Springer, 2012. ISBN 978-1489989673.

2. G. R. Grimmett, D. R. Stirzaker: Probability and Random Processes, 3rd Edition. Oxford University Press, 2001. ISBN 978-0198572220.

Note:

The course is taught in Czech. Classes are held in a 2+2 format during the first 10 weeks of the semester.

Further information:

https://courses.fit.cvut.cz/NI-NAH

No time-table has been prepared for this course

The course is a part of the following study plans:

• Master specialization Artificial Intelligence, in Czech, 2026 (compulsory elective course)