ECE 6437 – Computational Methods for Optimization

Course Objective

This course is designed to provide students with a thorough understanding of optimization concepts and methods for several important classes of nonlinear programming problems, as well as the implementation and testing of these methods in software. Our focus will be on robust methods that can solve practical and large problems. Extensions to discrete optimization will also be stressed.

Course Materials

Lecture 1 | Introduction, Necessary and Sufficient Conditions for Minima & Convex Analysis | ECE6437_Lecture01 ECE6437_Problemset1_Fall2009

Lecture 2 | Review, Contour Maps, Various Forms of Generalized Gradient Methods, and Line Search Methods | ECE6437_Lecture02 ECE6437_Problemset2_Fall2009

Lecture 3 | Quadratic Interpolation, Combined Golden Section Search and Quadratic Interpolation, Convergence of Generalized Gradient Method, Stopping Criteria | ECE6437_Lecture03 ECE6437_Problemset3_Fall2009

Lecture 4 | Newton’s Method and its Modifications | ECE6437_Lecture04 ECE6437_Problemset4_Fall2009

Lecture 5 | Conjugate Direction Methods, Convergence Analysis, Practicalities | ECE6437_Lecture05 ECE6437_Problemset5_Fall2009

Lecture 6 | Variable Metric (Secant, Quasi-Newton) Methods, Quadratic Termination, Practicalities, Incremental Methods for NN | ECE6437_Lecture06 ECE6437_Problemset6_Fall2009

Lecture 7 | Constrained Optimization: Necessary and Sufficient Conditions | ECE6437_Lecture07 ECE6437_Problemset7_Fall2009

Lecture 8 | Inequality (mixed) Constraints, Karusch-Kuhn-Tucker Conditions, Convex Programming,Primal-Dual Methods | ECE6437_Lecture08 ECE6437_Problemset8_Fall2009

Lecture 9-10 | Penalty and Augmented Lagrangian (Multiplier) Methods | ECE6437_Lectures9_10 ECE6437_Problemset9_Fall2009

Lecture 11 | Successive Quadratic Programming (SQP) Methods | ECE6437_Lecture11

Lecture 12 | Descent Methods for Constrained Minimization, Manifold Sub-optimization Methods, Problems with Simple Constraints |  ECE6437_Lecture12

Lecture 13 | Parallel Optimization Algorithms | ECE6437_Lecture13

Additional Material