Probability, Statistics, and Theory of Information
Code  Completion  Credits  Range  Language 

AE0B01PSI  Z,ZK  6  4+2 
 Lecturer:
 Tutor:
 Supervisor:
 Department of Mathematics
 Synopsis:

Basics of probability theory, mathematical statistics, information theory, and coding. Includes descriptions of probability, random variables and their distributions, characteristics and operations with random variables. Basics of mathematical statistics: Point and interval estimates, methods of parameters estimation and hypotheses testing, least squares method. Basic notions and results of the theory of Markov chains. Shannon entropy, mutual and conditional information.
 Requirements:

Linear Algebra, Calculus, Discrete Mathematics
 Syllabus of lectures:

1. Basic notions of probability theory. Kolmogorov model of probability. Independence, conditional probability, Bayes formula.
2. Random variables and their description. Random vector. Probability distribution function.
3. Quantile function. Mixture of random variables.
4. Characteristics of random variables and their properties. Operations with random variables.
Basic types of distributions.
5. Characteristics of random vectors. Covariance, correlation. Chebyshev inequality. Law of large numbers. Central limit theorem.
6. Basic notions of statistics. Sample mean, sample variance.
Interval estimates of mean and variance.
7. Method of moments, method of maximum likelihood. EM algorithm.
8. Hypotheses testing. Goodnessoffit tests, tests of correlation, nonparametic tests.
9. Discrete random processes. Stationary processes. Markov chains.
10. Classification of states of Markov chains.
11. Asymptotic properties of Markov chains. Overview of applications.
12. Shannon entropy. Entropy rate of a stationary information source.
13. Fundamentals of coding. Kraft inequality. Huffman coding.
14. Mutual information, capacity of an information channel.
 Syllabus of tutorials:

1. Elementary probability.
2. Kolmogorov model of probability. Independence, conditional probability, Bayes formula.
3. Mixture of random variables. Mean. Unary operations with random variables.
4. Dispersion (variance). Random vector, joint distribution. Binary operations with random variables.
5. Sample mean, sample variance. Chebyshev inequality. Central limit theorem.
6. Interval estimates of mean and variance.
7. Method of moments, method of maximum likelihood.
8. Hypotheses testing. Goodnessoffit tests, tests of correlation, nonparametic tests.
9. Discrete random processes. Stationary processes. Markov chains.
10. Classification of states of Markov chains.
11. Asymptotic properties of Markov chains.
12. Shannon entropy. Entropy rate of a stationary information source.
13. Fundamentals of coding. Kraft inequality. Huffman coding.
14. Mutual information, capacity of an information channel.
 Study Objective:

Basics of probability theory and their application in statistical estimates and tests.
The use of Markov chains in modeling.
Basic notions of information theory.
 Study materials:

[1] Papoulis, A.: Probability and Statistics, PrenticeHall, 1990.
[2] Stewart W.J.: Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling. Princeton University Press 2009.
[3] David J.C. MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
 Note:
 Further information:
 http://math.feld.cvut.cz/helisova/01pstimfe.html
 No timetable has been prepared for this course
 The course is a part of the following study plans:

 Cybernetics and Robotics  Robotics (compulsory course in the program)
 Cybernetics and Robotics  Senzors and Instrumention (compulsory course in the program)
 Cybernetics and Robotics  Systems and Control (compulsory course in the program)
 Electrical Engineering, Power Engineering and Management  Applied Electrical Engineering (elective course)
 Electrical Engineering, Power Engineering and Management  Electrical Engineering and Management (elective course)
 Communications, Multimedia and Electronics  Communication Technology (elective course)
 Communications, Multimedia and Electronics  Multimedia Technology (elective course)
 Communications, Multimedia and Electronics  Applied Electronics (elective course)
 Communications, Multimedia and Electronics  Network and Information Technology (elective course)
 Open Informatics  Computer Systems (compulsory course in the program)
 Open Informatics  Computer and Information Science (compulsory course in the program)
 Open Informatics  Software Systems (compulsory course in the program)
 Electrical Engineering, Power Engineering and Management (elective course)
 Communications, Multimedia and Electronics (elective course)
 Cybernetics and Robotics (compulsory course in the program)
 Open Informatics (compulsory course in the program)
 Communications, Multimedia and Electronics  Communications and Electronics (elective course)