Optimization Methods and Algorithms
Code  Completion  Credits  Range  Language 

01OPT  ZK  3  2+1  Czech 
 Garant předmětu:
 Lecturer:
 Tutor:
 Supervisor:
 Department of Mathematics
 Synopsis:

The course is devoted to various type of optimization techniques together with algorithms for employ them on real problems. It covers classical optimization techniques, numerical methods for optimization, linear programming, methods of nonlinear programming, dynamic programming, variation methods in statistics, and stochastic approximation.
 Requirements:

Basic course of Calculus and Linear Algebra Equations (in the extent of the courses 01MA1, 01MAB24, 01LA1, 01LAB2, 01NM held at the FNSPE CTU in Prague).
 Syllabus of lectures:

The aim of this course is to present mathematical models for various types of optimization techniques together with algorithms for employ them on real problems. Methods will be illustrated and applied on practical examples coming from probability and statistics (regress analysis, Markov processes, maximum likelihood estimate, etc.) 1. Classical optimization technique: generalized Lagrange method, Everett theorem, application: statistical multicomponent system, PCA method of main components, useful inequalities for optimization: classical, Kolmogorov, Chernoff, Kantorovich inequalities. Numerical optimization methods: Fisher scoring method, EM algorithm for noncomplete data. Linear programming: Dual problem for NeymanPearson generalized lemma. 2. Nonlinear programming methods: saddle point, KuhnTucker conditions, quadratic programming, convex programming, Čebyšev approximation, stochastic programming with probability bond, geometric programming, condensation method, application: regress model with bond, probability estimate of Markov chain, model with component variance in experiment design, estimate of parameters in distribution mixture. 3. Dynamic programming: Deterministic and stochastic management model, Bellman optimality principle, Pontrjagin maxima principle, application: deterministic managerial process, linear stochastic managerial process, cluster analysis, general Markov managerial process, Bayes solution for fish population estimation, sequence estimating, optimal experiment design. 4. Variation methods in statistics: EulerLagrange equations, sufficient condition for extreme, NeymanPearson technique, nonlinear momentum problem, maximal entropy principle, robust M and Lestimate, Fisher information, penalized estimate via maximal likelihood principle, WilcoxonMannWhitney test statistics. 5. Stochastic approximation techniques: Nonparametric iteration RobbinsMonroov procedure, distributional and general case, KieferWolfowitz approach, recursive estimating, and application on: best asymptotically normal estimate, random scan method, simulated annealingoptimization, optimality criterions in simulations.
 Syllabus of tutorials:

1.Classical optimization technique, EM algorithm, Linear programming. 2. Nonlinear programming, regress model with bond, probability estimate of Markov chain, model with component variance in experiment design, estimate of parameters in distribution mixture. 3. Dynamic programming: deterministic managerial process, linear stochastic managerial process, cluster analysis, general Markov managerial process, Bayes solution for fish population estimation, sequence estimating, optimal experiment design. 4. EulerLagrange equations, sufficient condition for extreme, NeymanPearson technique, nonlinear momentum problem, maximal entropy principle. 5. Best asymptotically normal estimate, random scan method, stimulated annealingoptimization, optimality criterions in simulations.
 Study Objective:

Classical optimization techniques, linear programming, nonlinear programming, dynamic programming, EulerLagrange equations, sufficient condition for extreme, stochastic approximation techniques.
 Study materials:

Key references:
[1] J.S. Rustagij, Optimization Techniques in Statistics, London Academic Press, Inc 1994.
Recommended references:
[2] J.Jahn, Introduction to the Tudory of Nonlinear Optimization, Berlin Springer 1996.
 Note:
 Further information:
 No timetable has been prepared for this course
 The course is a part of the following study plans: