Information Theory and Coding

Garant předmětu:

Lecturer:

Tutor:

Supervisor:

Department of Mathematics

Synopsis:

Fundamentals of information theory with a view towards efficient data compression and reliable transmission of information.

Requirements:

Probability, statistics, and information theory (A0B01PSI),

Discrete mathematics (A4B01DMA)

For more information see http://math.feld.cvut.cz/gollova/tik.html

Syllabus of lectures:

1) Entropy, divergence, mutual information. Conditional entropy and conditional information.

2) Entropy rate of stationary and ergodic sources.

3) Universal source coding. Lempel-Ziv coding.

4) Information channel. Basic types.

5) Channel capacity. Blahut-Arimoto algorithm.

6) Shannon theorem.

7) Introduction to secret sharing schemes.

8) Algebraic structures in error detection and correction. Groups and finite fields.

9) Linear codes. Hamming codes.

10) Polynomials over Z_p and quotient rings.

11) Cyclic codes.

12) Galois fields, primitive elements and field characteristics.

13) BCH codes.

14) Reserve.

Syllabus of tutorials:

1) Entropy, divergence, mutual information. Conditional entropy and conditional information.

2) Entropy rate of stationary and ergodic sources.

3) Universal source coding. Lempel-Ziv coding.

4) Information channel. Basic types.

5) Channel capacity. Blahut-Arimoto algorithm.

6) Shannon theorem.

7) Introduction to secret sharing schemes.

8) Algebraic structures in error detection and correction. Groups and finite fields.

9) Linear codes. Hamming codes.

10) Polynomials over Z_p and quotient rings.

11) Cyclic codes.

12) Galois fields, primitive elements and field characteristics.

13) BCH codes.

14) Reserve.

Study Objective:

Understanding of mathematical models used in coding and transmission of digital information.

Study materials:

[1] Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, 2006.

[2] Yeung, R.W.: Information Theory and Network Coding. Springer, 2008.

[3] Adámek, J.: Kódování. SNTL, Praha, 1989.

[4] Vajda, I.: Teorie informace. Vydavatelství ČVUT, 2004.

Note:

Further information:

http://math.feld.cvut.cz/gollova/tik.html

No time-table has been prepared for this course

The course is a part of the following study plans: