Statistics for Informatics

Course guarantor:

Lecturer:

Tutor:

Supervisor:

Department of Applied Mathematics

Synopsis:

The students will learn the basics of the probability theory, elements of information theory and stochastic processes, and some methods of computational statistics. They will understand the methods for statistical processing of large volumes of data. They will get skills in using computational methods and statistical software for these tasks.

Requirements:

Knowledge in differential and integral calculus, elementary knowledge in probability and statistics.

Syllabus of lectures:

1. Probability review: probability space, continuity of probability measure, conditional probability, Bayes theorem, independence of events.

2. Random variables and vectors: Independence, correlation, marginal, joint and conditional distributions, conditional expectation.

3. Weak and strong law of large numbers, Central Limit Theorem, condence intervals, statistical hypotheses testing.

4. Goodness-of-t tests, independence testing (chi-squared, runs above/below the mean, runs up/down), student's t-tests (single sample, paired, and independent samples).

5. Bootstrap-based condence intervals, studentized pivot; self-information, discrete Shannon entropy.

6. Joint and conditional entropy, mutual information, dierential Shannon entropy, estimation of entropy, kernel density estimates.

7. Random processes: Spectral density, stationarity, Gaussian random process, white noise.

8. Discrete-time Markov chains: Markov property, Chapman-Kolmogorov equation, stationarity, absorbing chains, birth and death chains.

9. Discrete-time Markov chains: Stopping times, strong Markov property, recurrent and transitional states, Limit theorems.

10. Queueing theory basics, Little's theorem, Poisson process, modeling customer arrival processes.

11. Spacial Poisson process, non-homogeneous Poisson process, queueing system M/G/innitn.

12. Monte Carlo methods: Monte Carlo estimates, Monte Carlo tests, reduction of variance.

13. Queueing systems M/M/1 and M/M/m; application in reliability: Kolmogorov equations for systems with a majority module and triple modular redundant systems.

Syllabus of tutorials:

1. Conditional probability, Bayes' theorem, decision trees.

2. Random variable, random vector, independent random variables.

3. Entropy and information of discrete random variable. Chain rule.

4. Entropy and information of continuous random variable.

5. Stochastic processes, autocorrelation function, cross-correlation function, spectral density.

6. Bernoulli and Poissonův process.

7. Markov processes with discrete and continuous time.

8. Applications of Monte Carlo method.

9. Generation of random numbers.

10. Bootstrap in statistical inference.

11. Estimation of probability density functions using parametric methods.

12. Nonparametric estimation of probability density functions.

13. Kernel estimators of probability density functions.

Study Objective:

The aim of the module is to provide an introduction to probability, information theory and stochastic processes. Furthermore, the module brings knowledge needed for data analysis and processing. It provides students with knowledge of computational methods and gets them acquainted with the use of statistical software.

Study materials:

1. Cover, T. M., Thomas, J. A. ''Elements of Information Theory (2nd Edition)''. Wiley-Interscience, 2006. ISBN 0471241954.

2. Gentle, J. E. ''Elements of Computational Statistics''. Springer, 2005. ISBN 0387954899.

3. Trivedi, K. S. ''Probability and Statistics with Reliability, Queueing, and Computer Science Applications (2nd Edition)''. Wiley-Interscience, 2001. ISBN 0471333417.

Note:

Further information:

https://courses.fit.cvut.cz/MIE-SPI/

No time-table has been prepared for this course

The course is a part of the following study plans: