1

Unconstrained Optimization Optimality Conditions (PDF)

2

Convex Unconstrained Optimization Optimality Conditions

3

Newton's Method (PDF)

4

Quadratic Forms (PDF)

5

Steepest Descent Method (PDF - 2.2 MB)

6

Constrained Optimization Optimality Conditions I (PDF)

7

Constrained Optimization Optimality Conditions II

8

Constrained Optimization Optimality Conditions III

9

Projection Methods for Equality Constrained Problems (PDF)

10

Projection Methods/Penalty Methods (PDF)

11

Penalty Methods

12

Barrier Methods, Conditional Gradient Method (PDF)

13

Midterm Exam

14

Interior-Point Methods for Linear Optimization I (PDF)

15

Interior-Point Methods for Linear Optimization II

16

Analysis of Convex Sets (PDF)

17

Analysis of Convex Functions

18

Duality Theory I (PDF)

19

Duality Theory II

20

Duality Theory III

21

Duality Theory IV (PDF)

22

Generalized Programming and Subgradient Optimization (PDF)

23

Semidefinite Optimization I (PDF)

24

Semidefinite Optimization II

25

Semidefinite Optimization III

26

Extensions and Wrap-up

1

The Basic Problem
Basic Definitions
Weirstrass Theorems
Necessary and Sufficient Conditions for Optimality

(PDF)

2

Newton's Method
When Newton's Method Fails
Rates of Convergence
Quadratic Forms
Eigenvectors/Eigenvalues/Decompositions

(PDF)

3

Method of Steepest Descent
Why this Method is Good
Why this Method is Bad
Line Search Algorithm

(PDF)

4

Separating Hyperplanes
Theorem of The Alternative (Farkas Lemma)
Necessary Conditions for Optimum of Constrained Problem
Finding Optima

(PDF)

5

When is KKT Necessary
Sufficient Conditions
Steepest Descent for Constrained Problems

(PDF)

6

Penalty/Barrier Methods
Quiz Review

(PDF)

10

Importance of Duality
Lagrangian Dual Approach
Features of The Dual
Column-Geometry Dual Approach
Weak Duality
Strong Duality

(PDF)