ČESKÉ VYSOKÉ UČENÍ TECHNICKÉ V PRAZE
STUDIJNÍ PLÁNY
2018/2019

# Probability, Statistics, and Theory of Information

Předmět není vypsán Nerozvrhuje se
Kód Zakončení Kredity Rozsah Jazyk výuky
AE0B01PSI Z,ZK 6 4+2
Přednášející:
Kateřina Helisová
Cvičící:
Kateřina Helisová
Předmět zajišťuje:
katedra matematiky
Anotace:

Basics of probability theory, mathematical statistics, information theory, a coding. Includes descriptions of probability, random variables and their distributions, characteristics and operations with random variables. Basics of mathematical statistics: Point and interval estimates, methods of parameters estimation and hypotheses testing, least squares method. Basic notions and results of the theory of Markov chains. Shannon entropy, mutual and conditional information, types of codes. Correspondence between entropy and the optimal code length. Information channels and their capacity, compression.

Linear Algebra, Calculus, Discrete Mathematics

Osnova přednášek:

1. Basic notions of probability theory. Random variables and their description.

2. Characteristics of random variables. Random vector, independence, conditional probability, Bayes formula.

3. Operations with random variables, mixture of random variables. Chebyshev inequality. Law of large numbers. Central limit theorem.

4. Basic notions of statistics. Sample mean, sample variance.

5. Method of moments, method of maximum likelihood. EM algorithm.

6. Interval estimates of mean and variance. Hypotheses testing.

7. Goodness-of-fit tests, tests of correlation, non-parametic tests.

8. Applications in decision-making under uncertainty and pattern recognition. Least squares method.

9. Discrete random processes. Stationary processes. Markov chains.

10. Classification of states of Markov chains. Overview of applications.

11. Shannon's entropy of a discrete distribution and its axiomatical formulation. Theorem on minimal and maximal entropy. Conditional entropy. Chain rule. Subadditivity. Entropy of a continuous variable.

12. Fano's inequality. Information of message Y in message X. Codes, prefix codes, nonsingular codes. Kraft-MacMillan's inequality.

13. Estimation of the average codelength by means of entropy. Huffman codes. Data compression using the law of large numbers. Typical messages. Entropy speed of stationary sources.

14. Information channel and its capacity. Basic types of information channels. Shannon's coding theorem. Universal compression. Ziv-Lempel codes.

Osnova cvičení:

1. Elementary probability. Random variables and their description.

2. Mean and variance of random variables. Unary operations with random variables.

3. Random vector, joint distribution.

4. Binary operations with random variables. Mixture of random variables. Central limit theorem.

5. Sample mean, sample variance. Method of moments, method of maximum likelihood.

6. Interval estimates of mean and variance.

7. Hypotheses testing.

8. Least squares method.

9. Goodness-of-fit tests.

10. Discrete random processes. Stationary processes. Markov chains.

11. Shannons's entropy of a discrete distribution and its axiomatical formulation. Theorem on minimal and maximal entropy. Conditional entropy. Chain rule. Subadditivity. Entropy of a continuous variable.

12. Fano's inequality. Information of message Y in message X. Codes, prefix codes, nonsingular codes. Kraft-MacMillan's inequality.

13. Estimation of the average codelength by means of entropy. Huffman codes. Data compression using the law of large numbers. Typical messages. Entropy speed of stationary sources.

14. Information channel and its capacity. Basic types of information channels. Shannon's coding theorem. Universal compression. Ziv-Lempel codes.

Cíle studia:

Zvládnutí základů teorie pravděpodobnosti a jejich využití pro statistické odhady a testy.

Využití Markovových řetězců pro modelování.

Základní pojmy teorie informace.

Studijní materiály:

[1] Papoulis, A.: Probability and Statistics, Prentice-Hall, 1990.

[2] Stewart W.J.: Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling. Princeton University Press 2009.

[3] David J.C. MacKay: Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.

Poznámka:

Podmínkou získání zápočtu je aktivní účast na cvičeních a vypracování Rozsah výuky v kombinované formě studia: 28p+6s

Další informace:
http://math.feld.cvut.cz/helisova/01pstimfe.html
Pro tento předmět se rozvrh nepřipravuje
Předmět je součástí následujících studijních plánů:
Platnost dat k 17. 7. 2019
Aktualizace výše uvedených informací naleznete na adrese http://bilakniha.cvut.cz/cs/predmet12773504.html