Stochastic Systems

The course is not on the list Without time-table
Code Completion Credits Range Language
01STOS ZK 2 2+0 Czech
Garant předmětu:
Department of Mathematics

The course is devoted to the theory of Markov processes as mathematical models for stochastic systems, i.e. dynamic systems influenced by randomness. The main goal consists in investigating the time limit behavior for different instances according the type of the system states. The models with discrete and continuous time are distinguished, an application for practical tasks is demonstrated, in particular for queuing systems.


Basic course of Calculus, Linear Algebra and Probability Theory (in the extent of the courses 01MA1, 01LA1, 01LAP, 01PRST held at the FNSPE

CTU in Prague).

Syllabus of lectures:

1 Stochastic dynamical systems, Markov processes, equilibrium, homogeneity, stationarity.

2 Markov chains, transition probability, recurrent and transient states.

3 Stationary distribution.

4 Hitting probabilities.

5 Examples: random walk, discrete time queuing model.

6 Simulation method Markov Chain Monte Carlo, probabilistic optimization algorithms, applications in statistical physics and image processing.

7 Markov processes with continuous time, transition rates.

8 Kolmogorov equations.

9 Poisson process, birth-and-death processes.

10 Queuing theory.

11 Queuing networks. Open and closed Jackson networks, computer and communication networks.

Syllabus of tutorials:
Study Objective:

The understanding of limit behavior of stochastic systems in the connection with the state classification. Skills: The construction of the transition probabilities matrix (transition rates) based on given information. Application of given methods in particular examples in physics and engineering.

Study materials:

Key references:

[1] Norris, J. R.. Markov Chains, Cambridge Uviversity Press 1997.

[2] Stroock, Daniel W.. An Introduction to Markov Processes, Springer 2005.

Recommended references:

[1] Nelson, Randolph. Probability, Stochastic Processes, and Queueing Theory, Springer 2005.

[2] Ching, Wai-Ki. Markov chains : models, algorithms and applications, Springer 2006.

Further information:
No time-table has been prepared for this course
The course is a part of the following study plans:
Data valid to 2024-05-23
Aktualizace výše uvedených informací naleznete na adrese https://bilakniha.cvut.cz/en/predmet12082205.html