Statistics for Informatics

Lecturer:

Tutor:

Supervisor:

Department of Applied Mathematics

Synopsis:

The students will learn the basics of the probability theory, elements of information theory and stochastic processes, and some methods of computational statistics. They will understand the methods for statistical processing of large volumes of data. They will get skills in using computational methods and statistical software for these tasks.

Requirements:

Knowledge in differential and integral calculus, elementary knowledge in probability and statistics.

Syllabus of lectures:

1. Probability review: probability space, continuity of probability measure, conditional probability, Bayes theorem, independence of events.

2. Random variables and vectors: Independence, correlation, marginal, joint and conditional distributions, conditional expectation.

3. Weak and strong law of large numbers, Central Limit Theorem, condence intervals, statistical hypotheses testing.

4. Goodness-of-t tests, independence testing (chi-squared, runs above/below the mean, runs up/down), student's t-tests (single sample, paired, and independent samples).

5. Bootstrap-based condence intervals, studentized pivot; self-information, discrete Shannon entropy.

6. Joint and conditional entropy, mutual information, dierential Shannon entropy, estimation of entropy, kernel density estimates.

7. Random processes: Spectral density, stationarity, Gaussian random process, white noise.

8. Discrete-time Markov chains: Markov property, Chapman-Kolmogorov equation, stationarity, absorbing chains, birth and death chains.

9. Discrete-time Markov chains: Stopping times, strong Markov property, recurrent and transitional states, Limit theorems.

10. Queueing theory basics, Little's theorem, Poisson process, modeling customer arrival processes.

11. Spacial Poisson process, non-homogeneous Poisson process, queueing system M/G/innitn.

12. Monte Carlo methods: Monte Carlo estimates, Monte Carlo tests, reduction of variance.

13. Queueing systems M/M/1 and M/M/m; application in reliability: Kolmogorov equations for systems with a majority module and triple modular redundant systems.

Syllabus of tutorials:

1. Conditional probability, Bayes' theorem, decision trees.

2. Random variable, random vector, independent random variables.

3. Entropy and information of discrete random variable. Chain rule.

4. Entropy and information of continuous random variable.

5. Stochastic processes, autocorrelation function, cross-correlation function, spectral density.

6. Bernoulli and Poissonův process.

7. Markov processes with discrete and continuous time.

8. Applications of Monte Carlo method.

9. Generation of random numbers.

10. Bootstrap in statistical inference.

11. Estimation of probability density functions using parametric methods.

12. Nonparametric estimation of probability density functions.

13. Kernel estimators of probability density functions.

Study Objective:

The aim of the module is to provide an introduction to probability, information theory and stochastic processes. Furthermore, the module brings knowledge needed for data analysis and processing. It provides students with knowledge of computational methods and gets them acquainted with the use of statistical software.

Study materials:

1. Shao, J. - Tu, D. The Jackknife and Bootstrap. Springer, 1995. ISBN 978-1-4612-0795-5.

2. Cover, T. M. - Thomas, J. A. Elements of Information Theory (2nd Edition). Wiley, 2006. ISBN 978-0-471-24195-9.

3. Ludeman, L. Random Processes: Filtering, Estimation, and Detection. Wiley{IEEE Press, 2003. ISBN 978-0-471-25975-6.

4. Durrett, R. Essentials of Stochastic Processes. Springer, 1999. ISBN 978-0387988368.

Note:

Further information:

No time-table has been prepared for this course

The course is a part of the following study plans:

• Knowledge Engineering, in Czech, Presented in Czech, for Students who Enrolled in 2015 (compulsory course in the program)
• Master Informatics, Presented in Czech, Version for Students who Enrolled in 2015 (compulsory course in the program)
• Knowledge Engineering, in Czech, Presented in Czech, Version 2016 and and 2017 (compulsory course in the program)
• Computer Security, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
• Computer Systems and Networks, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
• Design and Programming of Embedded Systems, in Czech, Version 2016 to 2019 (compulsory course in the program)
• Specialization Web and Software Engineering, in Czech, Version 2016 to 2019 (compulsory course in the program)
• Specialization Software Engineering, in Czech, Version 2016 to 2019 (compulsory course in the program)
• Specialization Web Engineering, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
• Master Informatics, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
• Specialization System Programming, Presented in Czech, Version 2016 to 2019 (compulsory course in the program)
• Specialization Computer Science, Presented in Czech, Version 2016-2017 (compulsory course in the program)
• Knowledge Engineering, in Czech, Presented in Czech, Version 2018 to 2019 (compulsory course in the program)