1

Introduction to dynamic programming; examples and formulation (PDF)

2

The dynamic programming algorithm (PDF)

3

Deterministic systems and the shortest path problem (PDF)

4

Shortest path algorithms (PDF)

5

Deterministic continuous-time optimal control (PDF)

6

Stopping and scheduling problems (PDF)

7

Linear systems with quadratic costs and inventory control (PDF)

8

Problems with imperfect state information (PDF)

9

Sufficient statistics (PDF)

10

Suboptimal control (PDF)

11

Rollout algorithms (PDF)

12

More on suboptimal control (PDF)

13

Infinite horizon I: stochastic shortest path problems (PDF)

14

Infinite horizon II: discounted problems (PDF)

15

Infinite horizon III: average cost problems (PDF)

16

Semi-Markov problems (PDF)

17

Infinite horizon: discounted problems I (PDF)

18

Infinite horizon: discounted problems II (PDF)

Midterm

19

Stochastic shortest path problems (PDF)

20

Overview of main approaches in approximate dynamic programming (PDF)

Detailed outline for approximate dynamic programming, lectures 20-25 (PDF)

21

Cost approximation: discounted cost (PDF)

22

Projected equation methods (PDF)

23

More on projected equations: Q-learning (PDF)

24

Extensions to stochastic shortest path and average cost (PDF)

25

Gradient methods for approximation in policy space (PDF)

26

Project presentations I

27

Project presentations II

Problem set 1 (PDF)

Problem set 2 (PDF)

Problem set 3 (PDF)

Problem set 4 (PDF)

Problem set 5 (PDF)

Problem set 6 (PDF)

Problem set 7 (PDF)

Problem set 8 (PDF)

Problem set 9 (PDF)