Physiology and modeling of hearing and vision

Garant předmětu:

Václav Vencovský

Lecturer:

Karel Fliegel, Miloš Klíma, Petr Maršálek, Václav Vencovský

Tutor:

Karel Fliegel, Miloš Klíma, František Rund, Václav Vencovský

Supervisor:

Department of Radioelectronics

Synopsis:

The primary aim of the course is to study the physiology of sensors and processes of perception of audio and visual information by human subjects as two central and most important communication channels, i.e., Human Auditory System (HAS) and Human Visual System (HVS).

The course summarizes current knowledge in the field of human vision and hearing physiology and, at the same time, presents their description using mathematical models using the latest computational tools and procedures, including Machine Learning (ML), Deep Learning (DL) and Artificial Intelligence (AI). Emphasis is also placed on current and prospective applications of the mentioned knowledge. The main application area is the audiovisual technology related to human perception, but the direct employment of the acquired knowledge also includes the areas of multimedia technology, control systems, automation, robotics, safety and security technology, bioinspired systems, etc. At the same time, students gain a general overview of information processing in biological systems. A separate part is the objectification of audiovisual information perceived quality, i.e., Quality of Experience (QoE).

The course is intended for students of master's degree in technical fields. The exercises will be devoted to fundamental experiments to determine the most important characteristics of HAS and HVS, including computational models and simulation of vision and hearing processes.

Requirements:

Signal processing and systems (Fourier transform, filtration, transfer function, impulse response, spectrum).

Syllabus of lectures:

1. Description of sound by objective quantities (frequency, intensity) vs. perception (pitch, loudness). Outer and middle ear (anatomy and physiology)

2. Inner ear (cochlea, Corti organ, mechanoelectric transformation, inner hair cells and outer hair cells)

3. Inner ear (active amplification of basilar membrane vibrations and their nonlinear response)

4. Auditory nerve, higher neural parts of the auditory pathway, audio coding in auditory nerve fibers and spatial hearing

5. Auditory cortex, sound perception

6. Hearing disorders and diagnostic methods

7. Sensory centers in the brain, interconnection of information from visual and auditory cortex

8. Basic physical principles of optics, photometry, colorimetry - summary

9. Biological evolution of eye and types of biological image sensors

10. Human Visual System (HVS) - anatomy of eye and optical path

11. Physiological laws in HVS, image processing in HVS, receptive fields and internal image coding

12. Visual illusions, visual disturbances, visual prostheses

13. Modeling of image processing in HVS - including Machine Learning (ML), Deep Learning (AI) and Artificial Intelligence (AI)

14. S Perceived audiovisual information quality, Quality of Experience (QoE), modelling of processes in Central Nervous System (CNS)

Syllabus of tutorials:

1. Introduction and explanation of practices

2. Methods in psychoacoustics (hearing threshold measurement, masking, loudness ...)

3. Objective methods of measurement of hearing system function (otoacoustic emissions, CAP, CM, ABR, EEG)

4. Function of outer and middle ear: computational models

5. Function of inner ear: computational models based on filter banks

6. Function of the inner ear: computational models considering the pressure field in the fluid inside the cochlea and the physical properties of the basilar membrane and the organ of Corti.

7. Computational models of mechanoelectric transduction in the inner hair cells.

8. Models of higher level of auditory pathway (cochlear nucleus, inferior colliculus, olivary system)

9. Viewing angle and direction dependency of the eye resolution

10. Trichromatic color perception - metamery, subjective colorimeter

11. CSF (Contrast Sensitivity Function) and identifying eye resolution ability, dependence on brightness

12. Simulation and modeling of image quality perception

13. Simulation and modeling of receptive images

14. Credit

Study Objective:

Study materials:

Basic literature:

1. Syka J., Vrabec F., Voldřich L.Fyziologie a patofyziologie zraku a sluchu, Avicenum 1981 (in czech only, it covers both vision and hearing, due to its earlier edition time, two additional books are given below)

2. Pickles J.O. An introduction to the physiology of hearing, Emerald Group Publishing Limited, Third edition, 2008.

3. Wandell B.A. Foundations of vision, Sinauer Associates, 1995.

Additional literature:

1. Manley G.A., Gummer A.W., Popper A.N., Fay R.R. (Eds.) Understanding the cochlea, Springer Handbook of Auditory Research, 2017

2. Lyon R.F. Human and machine hearing, Cambridge University Press, 2018

3. Kremers J. Human color vision, Springer Verlag, 2016

4. Zhaoping L. Understanding vision: Theory, models, and data, Oxford, 2018.

Note:

Further information:

https://moodle.fel.cvut.cz/courses/BE0M37FAV

Time-table for winter semester 2024/2025:

Time-table is not available yet

Time-table for summer semester 2024/2025:

Time-table is not available yet

The course is a part of the following study plans:

• Electronics and Communications - Audiovisual Technology and Signal Processing (compulsory elective course)
• Medical Electronics and Bioinformatics - Specialization Image Processing (compulsory elective course)
• Medical Electronics and Bioinformatics - Specialization Signal Processing (compulsory elective course)