Selected Topics in Optimization and Numerical mathematics

Course guarantor:

Štěpán Starosta

Lecturer:

Karel Klouda, Štěpán Starosta, Daniel Vašata

Tutor:

Karel Klouda, Štěpán Starosta, Daniel Vašata

Supervisor:

Department of Applied Mathematics

Synopsis:

The course focuses on optimization problems that appear in the field of machine learning and artificial intelligence. Students broaden their knowledge of continuous optimization obtained in the course Mathematics for informatics. The methods are explained and described along with the details on how they are implemented on computers. Hence, the relevant concepts of numerical matematics, mainly numerical linear algebra, are explained too.

Requirements:

NIE-MPI

Syllabus of lectures:

1. Continuous optimization: problem statement and machine learning examples.

2. - 3. (2) Iterative methods for finding local extremal values (gradient descent, Newton's method, and their variants).

4. Lagrange method, Karush–Kuhn–Tucker conditions.

5. Duality and interior point method.

6. - 7. (2) QR decomposition, algorithms computing QR decomposition, QR algorithm.

8. - 9. (2) Linear regression and least squares method: statistical and numerical properties.

10. - 11. (2) Support Vector Machines regression.

12. - 13. (2) Matrix factorizations and their usage in machine learning (SVD, PCA, non-negative factorization).

Syllabus of tutorials:

1. Iterative methods for local extrema

2. Constrained optimization

3. Duality

4. Matrix factorizations

5. SVD, PCA

6. SVM

Study Objective:

Study materials:

1. Christopher Bishop, Pattern Recognition and Machine Learning, Springer-Verlag New York, 2006

2. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, 2011.

3. Stephen Boyd, Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2004.

4. Lloyd N. Trefethen, David Bau, Numerical Linear Algebra, SIAM: Society for Industrial and Applied Mathematics, 1997

Note:

Further information:

https://courses.fit.cvut.cz/NI-PON

Time-table for winter semester 2024/2025:

Time-table is not available yet

Time-table for summer semester 2024/2025:

Time-table is not available yet

The course is a part of the following study plans:

• Master specialization Computer Security, in Czech, 2020 (elective course)
• Master specialization Design and Programming of Embedded Systems, in Czech, 2020 (elective course)
• Master specialization Computer Systems and Networks, in Czech, 202 (elective course)
• Master specialization Management Informatics, in Czech, 2020 (elective course)
• Master specialization Software Engineering, in Czech, 2020 (elective course)
• Master specialization System Programming, in Czech, version from 2020 (elective course)
• Master specialization Web Engineering, in Czech, 2020 (elective course)
• Master specialization Knowledge Engineering, in Czech, 2020 (PS)
• Master specialization Computer Science, in Czech, 2020 (elective course)
• Mgr. programme, for the phase of study without specialisation, ver. for 2020 and higher (VO, elective course)
• Master Specialization Digital Business Engineering, 2023 (VO)
• Master specialization System Programming, in Czech, version from 2023 (elective course)
• Master specialization Computer Science, in Czech, 2023 (elective course)