Statistics for Informatics

Course guarantor:

Lecturer:

Tutor:

Supervisor:

Department of Applied Mathematics

Synopsis:

The students will learn the basics of the probability theory, elements of information theory and stochastic processes, and some methods of computational statistics. They will understand the methods for statistical processing of large volumes of data. They will get skills in using computational methods and statistical software for these tasks.

Requirements:

Knowledge in differential and integral calculus, elementary knowledge in probability and statistics.

Syllabus of lectures:

1. Basics of Probability Theory: Probabilty Space, Definitions, Properties, Sigma-continuity, etc.

2. Basics of Probability Theory: Conditional Probability, Independence, Commented Examples

3. Basics of Probability Theory: Random Variables, Cumulative Distribution Function, Probability Density Function, Dependence, Random Vectors, Marginal and Joint Distribution

4. Basics of Probability Theory: Conditional Distribution, Conditional Expectation, Characteristics of Random Variables, Selected Examples of Probability Distributions

5. Basics of Probability Theory: Poisson Process, Simulation Methods, Generating Functions

6. Basics of Probability Theory: Strong Law of Large Numbers (SLLN), Central Limit Theorem (CLT), Large Deviations, Entropy

7. Discrete-time Markov Chains with Finite State Space: Basic Concepts, Irreducibility and Periodicity of States, Absorption Probability, Stopping Times

8. Discrete-time Markov Chains: Examples: generalized random walk, random walk on graph, gambler's ruin, coupon collector

9. Discrete-time Markov Chains: Asymptotic Stationarity, Uniqueness and Existence of Stationary Distributions, Convergence

10. Discrete-time Markov Chains: Branching Processes, Birth & Death processes

11. Monte Carlo Methods: Markov Chain Monte Carlo (MCMC) - Basic Concepts and Examples

12. Monte Carlo Methods: Fast convergence of MCMC, Propp-Wilson Algorithm, Sandwiching, Simulated Annealing

13. Monte Carlo Methods: Monte Carlo Estimates, Monte Carlo Tests, Reduction of Variance

14. Stochastic Processes: Definition, Distribution Function, Characteristics of Stochastic Processes

15. Stochastic Processes: Characteristics and Classification of Stochastic Processes, Examples

16. Basics of Queueing Theory: Elements of Queueing Systems, Request Arrival Process, Queueing Policy, Service Policy, Kendall Notation

17. Stochastic Processes: Application of the Poisson Process to Model Arrivals in Queueing Systems

18. Stochastic Processes: Application of the Poisson Process in Queueing Theory

19. Stochastic Processes: Non-homogeneous Poisson Process, Spatial Poisson Process, M/G/infinity Queue

20. Continuous-time Markov Chains: Jump Rates, Timing Jumps by Poisson Process Arrivals, Kolmogorov Equations

21. Basics of Queueing Theory: M/M/m Queues, Queueing Systems

22. Basics of Queueing Theory: Open and Closed Queueing Systems

23. Bootstrap Methods: Properties of Bootstrap Approximations, Bootstrap Correction of Estimation Bias

24. Bootstrap Methods: Bootstrap Confidence Intervals, Permutation Bootstrap

25. Bootstrap Methods: Bootstrap Confidence Intervals for Parameters in Linear Regression

26. Estimation of Probability Density Functions: Histogram, Kernel Estimates, Maximum Likelihood Estimation, Estimation by the Method of Moments

Syllabus of tutorials:

1. Conditional probability, Bayes' theorem, decision trees.

2. Random variable, random vector, independent random variables.

3. Entropy and information of discrete random variable. Chain rule.

4. Entropy and information of continuous random variable.

5. Stochastic processes, autocorrelation function, cross-correlation function, spectral density.

6. Bernoulli and Poissonův process.

7. Markov processes with discrete and continuous time.

8. Applications of Monte Carlo method.

9. Generation of random numbers.

10. Bootstrap in statistical inference.

11. Estimation of probability density functions using parametric methods.

12. Nonparametric estimation of probability density functions.

13. Kernel estimators of probability density functions.

Study Objective:

The aim of the module is to provide an introduction to probability, information theory and stochastic processes. Furthermore, the module brings knowledge needed for data analysis and processing. It provides students with knowledge of computational methods and gets them acquainted with the use of statistical software.

Study materials:

1. Cover, T. M., Thomas, J. A. ''Elements of Information Theory (2nd Edition)''. Wiley-Interscience, 2006. ISBN 0471241954.

2. Gentle, J. E. ''Elements of Computational Statistics''. Springer, 2005. ISBN 0387954899.

3. Trivedi, K. S. ''Probability and Statistics with Reliability, Queueing, and Computer Science Applications (2nd Edition)''. Wiley-Interscience, 2001. ISBN 0471333417.

Note:

Further information:

https://courses.fit.cvut.cz/MI-SPI/

No time-table has been prepared for this course

The course is a part of the following study plans: