MM Optimization Algorithms and Applications: Course Information

Find a surrogate function that majorizes the objective function. When the surrogate function is minimized, one can go downhill with respect to the objective function. The process is repeated.

Description

The MM principle provides a mechanism for creating optimization algorithms. If the objective function is to be minimized, the key idea is to compute a surrogate function that majorizes the objective function. The solution of the surrogate function minimization is used to compute the next surrogate function that majorizes the objective function. The process is continued until convergence. It has pervasive applications in many engineering application domains. The celebrated EM algorithm in computational statistics is a special case of the MM principle.

The main goal of the course is to introduce the theory of MM principle with an emphasis on related optimization algorithms and their applications to machine learning.

Intended Audience

PhD students in areas of applied mathematics, communication, control, computer sciences, networking, civil engineering.

Course Textbook

Kenneth Lange, MM Optimization Algorithms

Lectures

Schedule is available here and a summary of lectures is available here.

Grading

• Pass/Fail

• To pass the course, at least 70% of the grades have to be achieved.
  

• The course evaluation consists of

  • Homework (30%)

  • Take Home Exam (30%)

  • Mini Project (40%)

Course Learning Outcomes

After finishing the course, the attendant will

• Recognize the concept of MM Principle.

• Incorporate techniques for majorization and minorization into the design of MM optimization algorithms.

• Implement numerically the MM optimization algorithms in various application.

Working load

2h per Week + homework + one take home exam + mini-project report.

Credits

This is a 7 credit course.

Teaching and learning methodology

The lectures will be mainly based on blackboard and slides, see lectures.

Prerequisites

Background in elementary analysis, convex analysis, linear algebra, and statistics.