Selected Topics in Optimization and Numerical mathematics

Course guarantor:

Karel Klouda

Lecturer:

Karel Klouda, Štěpán Starosta, Daniel Vašata

Tutor:

Karel Klouda, Štěpán Starosta, Daniel Vašata

Supervisor:

Department of Applied Mathematics

Synopsis:

Students will be introduced to special optimization problems that arise in the field of machine learning and artificial intelligence and will extend the basic knowledge of continuous optimization acquired in previous studies. They will also learn about the details of implementing solutions to these problems on a computer and related mathematical concepts, especially from numerical linear algebra.

Requirements:

Syllabus of lectures:

1. Continuous optimization: problem statement and machine learning examples.

2. - 3. (2) Iterative methods for finding local extremal values (gradient descent, Newton's method, and their variants).

4. Lagrange method, KarushKuhnTucker conditions.

5. Duality and interior point method.

6. - 7. (2) QR decomposition, algorithms computing QR decomposition, QR algorithm.

8. - 9. (2) Linear regression and least squares method: statistical and numerical properties.

10. - 11. (2) Support Vector Machines regression.

12. - 13. (2) Matrix factorizations and their usage in machine learning (SVD, PCA, non-negative factorization).

Syllabus of tutorials:

1. Iterative methods for local extrema

2. Constrained optimization

3. Duality

4. Matrix factorizations

5. SVD, PCA

6. SVM

Study Objective:

Students will be introduced to special optimization problems that arise in the field of machine learning and artificial intelligence and will extend the basic knowledge of continuous optimization acquired in previous studies. They will also learn about the details of implementing solutions to these problems on a computer and related mathematical concepts, especially from numerical linear algebra.

Study materials:

1. Bishop, Ch.: Pattern Recognition and Machine Learning

Springer 2006

ISBN 978-0-387-31073-2

2. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction

Springer 2011

ISBN 978-0-387-84857-0

3. Boyd, S., Vandenberghe, L.: Convex Optimization

Cambridge University Press 2004

ISBN 9780521833783

4. Trefethen, L. N., Bau, D.: Numerical Linear Algebra

SIAM: Society for Industrial and Applied Mathematics 1997

ISBN 978-0-89871-361-9

5. Nocedal, J., Wright, S. W.: Numerical Optimization, 2nd Edition

Springer 2006

ISBN 978-0-387-40065-5

6. Strang, G.: Introduction to Linear Algebra, 5th Edition

Wellesley-Cambridge Press 2016

ISBN 978-0980232776

Note:

This course is presented in parallel with NI-PON in Czech language.

Information about the course and teaching materials can be found at https://courses.fit.cvut.cz/QNI-PON.

Further information:

https://courses.fit.cvut.cz/QNI-PON

No time-table has been prepared for this course

The course is a part of the following study plans:

• Quantum Informatics (compulsory elective course)