Statistics for Informatics
- Petr Novák
- Petr Novák
- Department of Applied Mathematics
The students will learn the basics of the probability theory, elements of information theory and stochastic processes, and some methods of computational statistics. They will understand the methods for statistical processing of large volumes of data. They will get skills in using computational methods and statistical software for these tasks.
Knowledge in differential and integral calculus, elementary knowledge in probability and statistics.
- Syllabus of lectures:
1. Probability review: probability space, continuity of probability measure, conditional probability, Bayes theorem, independence of events.
2. Random variables and vectors: Independence, correlation, marginal, joint and conditional distributions, conditional expectation.
3. Weak and strong law of large numbers, Central Limit Theorem, condence intervals, statistical hypotheses testing.
4. Goodness-of-t tests, independence testing (chi-squared, runs above/below the mean, runs up/down), student's t-tests (single sample, paired, and independent samples).
5. Bootstrap-based condence intervals, studentized pivot; self-information, discrete Shannon entropy.
6. Joint and conditional entropy, mutual information, dierential Shannon entropy, estimation of entropy, kernel density estimates.
7. Random processes: Spectral density, stationarity, Gaussian random process, white noise.
8. Discrete-time Markov chains: Markov property, Chapman-Kolmogorov equation, stationarity, absorbing chains, birth and death chains.
9. Discrete-time Markov chains: Stopping times, strong Markov property, recurrent and transitional states, Limit theorems.
10. Queueing theory basics, Little's theorem, Poisson process, modeling customer arrival processes.
11. Spacial Poisson process, non-homogeneous Poisson process, queueing system M/G/innitn.
12. Monte Carlo methods: Monte Carlo estimates, Monte Carlo tests, reduction of variance.
13. Queueing systems M/M/1 and M/M/m; application in reliability: Kolmogorov equations for systems with a majority module and triple modular redundant systems.
- Syllabus of tutorials:
1. Conditional probability, Bayes' theorem, decision trees.
2. Random variable, random vector, independent random variables.
3. Entropy and information of discrete random variable. Chain rule.
4. Entropy and information of continuous random variable.
5. Stochastic processes, autocorrelation function, cross-correlation function, spectral density.
6. Bernoulli and Poissonův process.
7. Markov processes with discrete and continuous time.
8. Applications of Monte Carlo method.
9. Generation of random numbers.
10. Bootstrap in statistical inference.
11. Estimation of probability density functions using parametric methods.
12. Nonparametric estimation of probability density functions.
13. Kernel estimators of probability density functions.
- Study Objective:
The aim of the module is to provide an introduction to probability, information theory and stochastic processes. Furthermore, the module brings knowledge needed for data analysis and processing. It provides students with knowledge of computational methods and gets them acquainted with the use of statistical software.
- Study materials:
1. Cover, T. M., Thomas, J. A. ''Elements of Information Theory (2nd Edition)''. Wiley-Interscience, 2006. ISBN 0471241954.
2. Gentle, J. E. ''Elements of Computational Statistics''. Springer, 2005. ISBN 0387954899.
3. Trivedi, K. S. ''Probability and Statistics with Reliability, Queueing, and Computer Science Applications (2nd Edition)''. Wiley-Interscience, 2001. ISBN 0471333417.
- Further information:
- Time-table for winter semester 2020/2021:
- Time-table is not available yet
- Time-table for summer semester 2020/2021:
- The course is a part of the following study plans:
- Computer Security, Presented in English, Version 2016 až 2020 (compulsory course in the program)
- Computer Systems and Networks, Presented in English, Version 2016 až 2020 (compulsory course in the program)
- Design and Programming of Embedded Systems, in English, Version 2016 až 2020 (compulsory course in the program)
- Specialization Software Engineering, in English, Version 2016 až 2020 (compulsory course in the program)