Stochastic Systems
Code  Completion  Credits  Range 

D01STOS  ZK 
 Garant předmětu:
 Lecturer:
 Tutor:
 Supervisor:
 Department of Mathematics
 Synopsis:

The course is devoted to the theory of Markov processes as mathematical models for stochastic systems, i.e. dynamic systems influenced by randomness. The main goal consists in investigating the time limit behavior for different instances according the type of the system states. The models with discrete and continuous time are distinguished, an application for practical tasks is demonstrated, in particular for queuing systems.
 Requirements:

Basic course of Calculus, Linear Algebra and Probability Theory (in the extent of the courses 01MA1, 01LA1, 01LAP, 01PRST held at the FNSPE
CTU in Prague).
 Syllabus of lectures:

1 Stochastic dynamical systems, Markov processes, equilibrium, homogeneity, stationarity.
2 Markov chains, transition probability, recurrent and transient states.
3 Stationary distribution.
4 Hitting probabilities.
5 Examples: random walk, discrete time queuing model.
6 Simulation method Markov Chain Monte Carlo, probabilistic optimization algorithms, applications in statistical physics and image processing.
7 Markov processes with continuous time, transition rates.
8 Kolmogorov equations.
9 Poisson process, birthanddeath processes.
10 Queuing theory.
11 Queuing networks. Open and closed Jackson networks, computer and communication networks.
 Syllabus of tutorials:
 Study Objective:

The understanding of limit behavior of stochastic systems in the connection with the state classification. Skills: The construction of the transition probabilities matrix (transition rates) based on given information. Application of given methods in particular examples in physics and engineering.
 Study materials:

Key references:
[1] Norris, J. R.. Markov Chains, Cambridge Uviversity Press 1997.
[2] Stroock, Daniel W.. An Introduction to Markov Processes, Springer 2005.
Recommended references:
[1] Nelson, Randolph. Probability, Stochastic Processes, and Queueing Theory, Springer 2005.
[2] Ching, WaiKi. Markov chains : models, algorithms and applications, Springer 2006.
 Note:
 Further information:
 No timetable has been prepared for this course
 The course is a part of the following study plans: