Course Objective
This course is designed to provide students with a thorough understanding of optimization concepts and methods for several important classes of nonlinear programming problems, as well as the implementation and testing of these methods in software. Our focus will be on robust methods that can solve practical and large problems. Extensions to discrete optimization will also be stressed.
Course Materials
Lecture 1 | Introduction, Necessary and Sufficient Conditions for Minima & Convex Analysis | ECE6437_Lecture01 ECE6437_Problemset1_Fall2009
Lecture 2 | Review, Contour Maps, Various Forms of Generalized Gradient Methods, and Line Search Methods | ECE6437_Lecture02 ECE6437_Problemset2_Fall2009
Lecture 3 | Quadratic Interpolation, Combined Golden Section Search and Quadratic Interpolation, Convergence of Generalized Gradient Method, Stopping Criteria | ECE6437_Lecture03 ECE6437_Problemset3_Fall2009
Lecture 4 | Newton’s Method and its Modifications | ECE6437_Lecture04 ECE6437_Problemset4_Fall2009
Lecture 5 | Conjugate Direction Methods, Convergence Analysis, Practicalities | ECE6437_Lecture05 ECE6437_Problemset5_Fall2009
Lecture 6 | Variable Metric (Secant, Quasi-Newton) Methods, Quadratic Termination, Practicalities, Incremental Methods for NN | ECE6437_Lecture06 ECE6437_Problemset6_Fall2009
Lecture 7 | Constrained Optimization: Necessary and Sufficient Conditions | ECE6437_Lecture07 ECE6437_Problemset7_Fall2009
Lecture 8 | Inequality (mixed) Constraints, Karusch-Kuhn-Tucker Conditions, Convex Programming,Primal-Dual Methods | ECE6437_Lecture08 ECE6437_Problemset8_Fall2009
Lecture 9-10 | Penalty and Augmented Lagrangian (Multiplier) Methods | ECE6437_Lectures9_10 ECE6437_Problemset9_Fall2009
Lecture 11 | Successive Quadratic Programming (SQP) Methods | ECE6437_Lecture11
Lecture 12 | Descent Methods for Constrained Minimization, Manifold Sub-optimization Methods, Problems with Simple Constraints | ECE6437_Lecture12
Lecture 13 | Parallel Optimization Algorithms | ECE6437_Lecture13