MATH 309-3
Continuous Optimization

Spring 2014

INSTRUCTOR: John Stockie
Office: K 10518
Phone: 778-782-3553
E-mail: stockie  [at]
CLASS TIMES: MWF    9:30-10:20,   WMC 3210
MY OFFICE HOURS: Fridays    10:30-11:20
TUTORIALS: Each of you is assigned to a tutorial section which is run by your TA:

Lee Safranek (email: ljs8 [at] sfu [dot] ca)
Tuesdays - 16:30-17:20,   AQ 5030 (D101)
13:30-14:20, AQ 5037 (D102)

I strongly encourage you to attend your scheduled tutorial session. Your TA is available to provide help with homework questions or explaining material from class. He will also review homework and midterm solutions. When needed, your TA may provide supplementary material and additional examples that I couldn't otherwise cover in class.
PREREQUISITES: MATH 240 or 232, and MATH 251. Recommended: MATH 308.
TEXTBOOK: The required textbook for the course is:
    Numerical Optimization, 2nd edition, by J. Nocedal and S.J. Wright (Springer, 2006)

This book has an electronic version that you can access via the SFU Library (just click on the link above). Another excellent book that I recommend for ``extra reading'' is also available in the library as a hard-copy:
    The Mathematics of Nonlinear Programming, by A.L. Peressini, F.E. Sullivan and J.J. Uhl (Springer, 1998)
 Both textbooks are held on reserve in the library.
Homework will be assigned roughly every second week and you will have approximately two weeks to complete each assignment. Homework is due on Fridays at 12:00 noon and should be deposited in the appropriate locked submission box on the 9000 level below the Math Department. Any late assignment will receive a mark of zero, with no exceptions -- to handle any unforseen circumstances, I will drop everyone's lowest homework grade when calculating your final marks.
Some homework questions will be taken from the textbook, with additional problems assigned from other sources.
Some computer programming will be required on assignments, and the language I will use is Matlab, which is available to everyone on the university computing network. Although you are expected to be able to write your own Matlab code from scratch, the length of the code required will actually be quite short. To help you becoming familiar with Matlab (if you aren't already) I will provide many samples of Matlab code from examples in class, and many assignment questions will involve modifying an existing Matlab code that I will provide to you.
TESTS: There will be one midterm test held in class on Wednesday February 26. The final exam is scheduled for Saturday April 12 at 12:00pm and will cover all material in the course.
Academic dishonesty has no place in a university and I have zero tolerance for it. All students must understand the meaning and consequences of cheating, plagiarism and other academic offences identified under the SFU Code of Academic Integrity and Good Conduct. Cheating includes, but is not limited to:
  • Handing in assignment solutions copied from other sources such as solution manuals, other students' work, on-line sources, etc.
  • Using calculators or unauthorized reference materials during tests or examinations, unless they are explicitly allowed.
  • Looking at the work of other students during examinations.
In all of these cases, all students involved in the act will receive a mark of zero for the entire work in question. The Chair of the Mathematics Department will be notified and a permanent note will go in your academic file. Further action may also be taken as outlined in the SFU Policies and Procedures for Student Discipline.
OUTLINE: (below is the official outline for Spring 2014, which may differ from what is posted on the Math Department's web site)
This course covers elementary aspects of the theory, numerical algorithms, and practical applications of convex optimization, which refers to finding maxima/minima of a function of several real variables with/without (in)equality constraints. Examples will be drawn from applications in scheduling and assignment, image reconstruction, signal processing, economics, and engineering design. An essential component of this course will be understanding optimization algorithms (mostly implemented in Matlab) and using them to solve real problems.

Below is a rough outline of topics to be covered along with pointers to relevant sections of the textbook. Additional examples and material will be added as required from the text and other sources.

  1. Review of basic concepts:
    • Elements of multivariable calculus (analysis and topology) (App. A)
    • Elements of linear algebra (App. A)
    • Introduction to Matlab

  2. Unconstrained optimization:
    • Necessary and sufficient conditions for a minimum (Chs. 1-2)
    • Line search methods (Ch. 3)
    • Conjugate gradient methods (Ch. 5)
    • Newton-type methods (Sec. 7.1)
    • Quasi-Newton methods (Chs. 6-7)

  3. Constrained optimization:
    • First- and second-order optimality (KKT) conditions (Ch. 12)
    • Quadratic programming (Ch. 16)
    • Penalty, barrier and augmented-Lagrangian methods (Ch. 17)
    • Sequential quadratic programming (Ch. 18)

   Assignments (bi-weekly, best 5/6):   25%
Midterm test (Feb. 26):   25%
Final examination (Apr. 12):    50%
Students requesting any special accommodations for reasons of disability, religion, etc. MUST inform me during the first week of the semester.

Last modified: Tue Jan 14 2014