Information Theory and Coding

Course guarantor:

Lecturer:

Tutor:

Supervisor:

Department of Mathematics

Synopsis:

Fundamentals of information theory with a view towards efficient data compression and reliable transmission of information using selfcorrecting codes.

Requirements:

Probability and statistics

Discrete mathematics

Syllabus of lectures:

1) Algebraic structures in error detection and correction. Countimg modulo n.

2) Linear algebra over field Zp.

3) Linear codes - generating and controling matrix.

4) Error correction, Hamming codes.

5) Polynomials over Zp and quotient rings of polynomials.

6) Cyclic codes - generating and controling polynomial.

7) Galois fields, primitive element.

8) Generating roots of cyclic codes in a field.

9) BCH codes.

10) Information theory - probability and entropy.

11) Entropy, divergence, mutual information.

12) Data compression and source coding.

13) Universal source coding (Lempel - Ziv).

14) Information channel. Shannon theorem about capacity of channel.

Syllabus of tutorials:

1) Algebraic structures in error detection and correction. Countimg modulo n.

2) Linear algebra over field Zp.

3) Linear codes - generating and controling matrix.

4) Error correction, Hamming codes.

5) Polynomials over Zp and quotient rings of polynomials.

6) Cyclic codes - generating and controling polynomial.

7) Galois fields, primitive element.

8) Generating roots of cyclic codes in a field.

9) BCH codes.

10) Information theory - probability and entropy.

11) Entropy, divergence, mutual information.

12) Data compression and source coding.

13) Universal source coding (Lempel - Ziv).

14) Information channel. Shannon theorem about capacity of channel.

Study Objective:

Understanding of mathematical models used in coding and transmission of digital information.

Study materials:

[1] Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, 2006.

[2] Yeung, R.W.: Information Theory and Network Coding. Springer, 2008.

[3] Adámek, J.: Kódování. SNTL, Praha, 1989.

[4] Vajda, I.: Teorie informace. Vydavatelství ČVUT, 2004.

Note:

Further information:

https://math.fel.cvut.cz/en/people/gollova/tik.html

No time-table has been prepared for this course

The course is a part of the following study plans:

• Electronics and Communications (compulsory elective course)