Statistics for Informatics
Code | Completion | Credits | Range | Language |
---|---|---|---|---|
MIE-SPI.1 | Z,ZK | 8 | 4P+2C | English |
- Garant předmětu:
- Lecturer:
- Tutor:
- Supervisor:
- Department of Applied Mathematics
- Synopsis:
-
The students will learn the basics of the probability theory, elements of information theory and stochastic processes, and some methods of computational statistics. They will understand the methods for statistical processing of large volumes of data. They will get skills in using computational methods and statistical software for these tasks.
- Requirements:
-
Knowledge in differential and integral calculus, elementary knowledge in probability and statistics.
- Syllabus of lectures:
-
1. Basics of Probability Theory: Probabilty Space, Definitions, Properties, Sigma-continuity, etc.
2. Basics of Probability Theory: Conditional Probability, Independence, Commented Examples
3. Basics of Probability Theory: Random Variables, Cumulative Distribution Function, Probability Density Function, Dependence, Random Vectors, Marginal and Joint Distribution
4. Basics of Probability Theory: Conditional Distribution, Conditional Expectation, Characteristics of Random Variables, Selected Examples of Probability Distributions
5. Basics of Probability Theory: Poisson Process, Simulation Methods, Generating Functions
6. Basics of Probability Theory: Strong Law of Large Numbers (SLLN), Central Limit Theorem (CLT), Large Deviations, Entropy
7. Discrete-time Markov Chains with Finite State Space: Basic Concepts, Irreducibility and Periodicity of States, Absorption Probability, Stopping Times
8. Discrete-time Markov Chains: Examples: generalized random walk, random walk on graph, gambler's ruin, coupon collector
9. Discrete-time Markov Chains: Asymptotic Stationarity, Uniqueness and Existence of Stationary Distributions, Convergence
10. Discrete-time Markov Chains: Branching Processes, Birth & Death processes
11. Monte Carlo Methods: Markov Chain Monte Carlo (MCMC) - Basic Concepts and Examples
12. Monte Carlo Methods: Fast convergence of MCMC, Propp-Wilson Algorithm, Sandwiching, Simulated Annealing
13. Monte Carlo Methods: Monte Carlo Estimates, Monte Carlo Tests, Reduction of Variance
14. Stochastic Processes: Definition, Distribution Function, Characteristics of Stochastic Processes
15. Stochastic Processes: Characteristics and Classification of Stochastic Processes, Examples
16. Basics of Queueing Theory: Elements of Queueing Systems, Request Arrival Process, Queueing Policy, Service Policy, Kendall Notation
17. Stochastic Processes: Application of the Poisson Process to Model Arrivals in Queueing Systems
18. Stochastic Processes: Application of the Poisson Process in Queueing Theory
19. Stochastic Processes: Non-homogeneous Poisson Process, Spatial Poisson Process, M/G/infinity Queue
20. Continuous-time Markov Chains: Jump Rates, Timing Jumps by Poisson Process Arrivals, Kolmogorov Equations
21. Basics of Queueing Theory: M/M/m Queues, Queueing Systems
22. Basics of Queueing Theory: Open and Closed Queueing Systems
23. Bootstrap Methods: Properties of Bootstrap Approximations, Bootstrap Correction of Estimation Bias
24. Bootstrap Methods: Bootstrap Confidence Intervals, Permutation Bootstrap
25. Bootstrap Methods: Bootstrap Confidence Intervals for Parameters in Linear Regression
26. Estimation of Probability Density Functions: Histogram, Kernel Estimates, Maximum Likelihood Estimation, Estimation by the Method of Moments
- Syllabus of tutorials:
-
1. Conditional probability, Bayes' theorem, decision trees.
2. Random variable, random vector, independent random variables.
3. Entropy and information of discrete random variable. Chain rule.
4. Entropy and information of continuous random variable.
5. Stochastic processes, autocorrelation function, cross-correlation function, spectral density.
6. Bernoulli and Poissonův process.
7. Markov processes with discrete and continuous time.
8. Applications of Monte Carlo method.
9. Generation of random numbers.
10. Bootstrap in statistical inference.
11. Estimation of probability density functions using parametric methods.
12. Nonparametric estimation of probability density functions.
13. Kernel estimators of probability density functions.
- Study Objective:
-
The aim of the module is to provide an introduction to probability, information theory and stochastic processes. Furthermore, the module brings knowledge needed for data analysis and processing. It provides students with knowledge of computational methods and gets them acquainted with the use of statistical software.
- Study materials:
-
1. Cover, T. M., Thomas, J. A. ''Elements of Information Theory (2nd Edition)''. Wiley-Interscience, 2006. ISBN 0471241954.
2. Gentle, J. E. ''Elements of Computational Statistics''. Springer, 2005. ISBN 0387954899.
3. Trivedi, K. S. ''Probability and Statistics with Reliability, Queueing, and Computer Science Applications (2nd Edition)''. Wiley-Interscience, 2001. ISBN 0471333417.
- Note:
- Further information:
- https://courses.fit.cvut.cz/MIE-SPI/
- No time-table has been prepared for this course
- The course is a part of the following study plans: