Welcome to EECS 127/227A!
Lecture recordings from this semester are available here. Lecture recordings from a previous semester (Fall 2022) are available here. They cover a lot of the current course material, but neither set of material is a strict superset of the other.
Previous semesters’ staff have written a course reader (i.e., set of course notes), accessible here. This semester’s staff will periodically update and improve the reader.
Readings posted next to each lecture use the abbreviations:
- CEG, which stands for the textbook Optimization Models by Calafiore and El Ghaoui;
- BV, which stands for the textbook Convex Optimization by Boyd and Vandenberghe.
Staff
Syllabus
Course Calendar
Note: calendar not finalized and subject to change. In particular, the schedule of certain topics still have to be finalized.
Jump to current week.
Week 1
| Jan 15: | ||
|---|---|---|
| Jan 16: | [LECTURE 1] Introduction and Least Squares [HOMEWORK 1 RELEASED] Prob. PDF |
CEG Ch. 1 |
| Jan 17: | ||
| Jan 18: | [LECTURE 2] Linear Algebra Review: Vector Norms | CEG Ch. 2, Ch. 3 |
| Jan 19: | [DISCUSSION 1] Prob. PDF - Sol. PDF [HOMEWORK 1 DUE] Sol. PD [HOMEWORK 1 SELF-GRADES RELEASED] Gradescope [HOMEWORK 2 RELEASED] Prob. PDF |
Week 2
| Jan 22: | [DISCUSSION 1] Prob. PDF - Sol. PDF | |
|---|---|---|
| Jan 23: | [LECTURE 3] Linear Algebra Review: Gram-Schmidt, QR, Fundamental Theorem of Linear Algebra, Minimum-Norm Solution | CEG Ch. 4 |
| Jan 24: | ||
| Jan 25: | [LECTURE 4] Linear Algebra Review: Symmetric Matrices | CEG Ch. 5 |
| Jan 26: | [DISCUSSION 2] Prob. PDF - Sol. PDF [HOMEWORK 1 SELF-GRADES DUE] [HOMEWORK 2 DUE] Sol. PDF [HOMEWORK 2 SELF-GRADES RELEASED] Gradescope [HOMEWORK 3 RELEASED] Prob. PDF |
Week 3
| Jan 29: | [DISCUSSION 2] Prob. PDF - Sol. PDF | |
|---|---|---|
| Jan 30: | [LECTURE 5] Linear Algebra Review: Principal Components Analysis, SVD - Jupyter Notebook | CEG Ch. 5 |
| Jan 31: | ||
| Feb 1: | [LECTURE 6] SVD, Low-Rank Approximation I | CEG Sec. 5.3 |
| Feb 2: | [DISCUSSION 3] Prob. PDF - Sol. PDF [HOMEWORK 2 SELF-GRADES DUE] [HOMEWORK 3] DUE Sol. PDF [HOMEWORK 3 SELF-GRADES RELEASED] Gradescope [HOMEWORK 4 RELEASED] Prob. PDF - Prob. Jupyter |
Week 4
| Feb 5: | [DISCUSSION 3] Prob. PDF - Sol. PDF | |
|---|---|---|
| Feb 6: | [LECTURE 7] Low-Rank Approximation II | |
| Feb 7: | ||
| Feb 8: | [LECTURE 8] Vector Calculus I | BV Appendix A.4 |
| Feb 9: | [DISCUSSION 4] Prob. PDF [HOMEWORK 3 SELF-GRADES DUE] [HOMEWORK 4] DUE Sol. PDF - Sol. Jupyter [HOMEWORK 4 SELF-GRADES RELEASED] Gradescope [HOMEWORK 5 RELEASED] Prob. PDF |
Week 5
| Feb 12: | [DISCUSSION 4] Prob. PDF - Sol. PDF | |
|---|---|---|
| Feb 13: | [LECTURE 9] Vector Calculus II | BV Appendix A.4 |
| Feb 14: | ||
| Feb 15: | LECTURE 10 Least Squares and Variants: Ridge Regression | CEG Ch. 6 |
| Feb 16: | [DISCUSSION 5] Prob. PDF - Prob. Jupyter - Sol. PDF [HOMEWORK 4 SELF-GRADES DUE] [HOMEWORK 5] DUE Sol. PDF [HOMEWORK 5 SELF-GRADES RELEASED] Gradescope [HOMEWORK 6 RELEASED] Prob. PDF |
Week 6
| Feb 19: | [DISCUSSION 5] Prob. PDF - Prob. Jupyter - Sol. PDF | |
|---|---|---|
| Feb 20: | [LECTURE 11] Convexity I | CEG Sec. 8.1, 8.2, 8.3 BV Ch. 2, 3, 4 |
| Feb 21: | ||
| Feb 22: | [LECTURE 12] Convexity II | CEG Sec. 8.1, 8.2, 8.3 BV Ch. 2, 3, 4 |
| Feb 23: | [DISCUSSION 6] Prob. PDF - Sol. PDF [HOMEWORK 5 SELF-GRADES DUE] [HOMEWORK 6] DUE Sol. PDF [HOMEWORK 6 SELF-GRADES RELEASED] Gradescope [HOMEWORK 7 RELEASED] Prob. PDF - Sol. PDF |
Week 7
| Feb 26: | [DISCUSSION 6] Prob. PDF - Sol. PDF | |
|---|---|---|
| Feb 27: | [LECTURE 13] Descent Methods I | CEG Sec. 12.2 |
| Feb 28: | ||
| Feb 29: | [LECTURE 14] Descent Methods II | CEG Sec. 12.2 |
| Mar 1: | [DISCUSSION 7] Prob. PDF - Sol. PDF [HOMEWORK 6 SELF-GRADES DUE] [HOMEWORK 7 DUE] [HOMEWORK 7 SELF-GRADES RELEASED] |
Week 8
| Mar 4: | [DISCUSSION 7] Prob. PDF - Sol. PDF [MIDTERM EXAM] |
|
|---|---|---|
| Mar 5: | [LECTURE 15] TBD [MIDTERM REDO RELEASED] Prob. PDF |
|
| Mar 6: | ||
| Mar 7: | [LECTURE 16] Weak Duality | CEG Sec. 8.5 BV Ch. 5 |
| Mar 8: | [DISCUSSION 8] Prob. PDF - Sol. PDF [MIDTERM REDO DUE] Sol. PDF [HOMEWORK 7 SELF-GRADES DUE] Gradescope [HOMEWORK 8 RELEASED] Prob. PDF - Prob. Jupyter |
Week 9
| Mar 11: | [DISCUSSION 8] Prob. PDF - Sol. PDF | |
|---|---|---|
| Mar 12: | [LECTURE 17] Strong Duality | CEG Sec. 8.5 BV Ch. 5 |
| Mar 13: | ||
| Mar 14: | [LECTURE 18] Duality, Optimality Conditions | BV Sec. 5.5 |
| Mar 15: | [DISCUSSION 9] Prob. PDF - Sol. PDF [HOMEWORK 8 DUE] Sol. PDF - Sol. Jupyter [HOMEWORK 8 SELF-GRADES RELEASED] Gradescope [HOMEWORK 9 RELEASED] Prob. PDF - Prob. Jupyter |
Week 10
| Mar 18: | [DISCUSSION 9] Prob. PDF - Sol. PDF | |
|---|---|---|
| Mar 19: | [LECTURE 19] KKT, Formulating Optimization Problems | BV Sec. 5.5 |
| Mar 20: | ||
| Mar 21: | [LECTURE] 20 LPs | CEG Ch. 9 |
| Mar 22: | [DISCUSSION 10] Prob. PDF - Sol. PDF [HOMEWORK 8 SELF-GRADES DUE] [HOMEWORK 9 DUE] Sol. PDF - Sol. Jupyter [HOMEWORK 9 SELF-GRADES RELEASED] Gradescope [HOMEWORK 10 RELEASED] Prob. PDF |
Week 11 (Spring Break)
| Mar 25: | ||
|---|---|---|
| Mar 26: | ||
| Mar 27: | ||
| Mar 28: | ||
| Mar 29: |
Week 12
| Apr 1: | [DISCUSSION 10] Prob. PDF - Sol. PDF | |
|---|---|---|
| Apr 2: | [LECTURE 21] QPs | CEG Ch. 9, 10 |
| Apr 3: | ||
| Apr 4: | [LECTURE 22] SOCPs | CEG Ch. 12 |
| Apr 5: | [DISCUSSION 11] Prob. PDF - Sol. PDF [HOMEWORK 9 SELF-GRADES DUE] [HOMEWORK 10 DUE] Sol. PDF [HOMEWORK 10 SELF-GRADES RELEASED] Gradescope [HOMEWORK 11 RELEASED] Prob. PDF - Prob. Jupyter |
Week 13
| Apr 8: | [DISCUSSION 11] Prob. PDF - Sol. PDF | |
|---|---|---|
| Apr 9: | [LECTURE 23] L1 Norms and LASSO | CEG Sec. 9.6.2, 13.4, 12.5 |
| Apr 10: | ||
| Apr 11: | [LECTURE 24] Advanced Descent Methods | CEG Sec. 12.5 BV Ch. 11 (first half) |
| Apr 12: | [DISCUSSION 12] Prob. PDF - Sol. PDF [HOMEWORK 10 SELF-GRADES DUE] [HOMEWORK 11 DUE] Sol. PDF - Sol. Jupyter [HOMEWORK 11 SELF-GRADES RELEASED] Gradescope [HOMEWORK 12 RELEASED] Prob. PDF - Prob. Jupyter |
Week 14
| Apr 15: | [DISCUSSION 12] Prob. PDF - Sol. PDF |
|---|---|
| Apr 16: | [LECTURE 25] Applications: SVM |
| Apr 17: | |
| Apr 18: | [LECTURE 26] Applications: SVM |
| Apr 19: | [DISCUSSION 13] Prob. PDF - Sol. PDF [HOMEWORK 11 SELF-GRADES DUE] [HOMEWORK 12 DUE] Sol. PDF - Sol. Jupyter [HOMEWORK 12 SELF-GRADES RELEASED] Gradescope [HOMEWORK 13 RELEASED] Prob. PDF - Prob. Jupyter |
Week 15
| Apr 22: | [DISCUSSION 13] Prob. PDF - Sol. PDF |
|---|---|
| Apr 23: | [LECTURE 27] Guest Lecture: Semidefinite Programming (Venkat Anantharam) |
| Apr 24: | |
| Apr 25: | [LECTURE 28] Guest Lecture: Producing (almost) separating hyperplanes – geometry,perceptrons, multiplicative weights (Satish Rao) |
| Apr 26: | [HOMEWORK 12 SELF-GRADES DU] [HOMEWORK 13 DUE] Sol. PDF - Sol. Jupyter [HOMEWORK 13 SELF-GRADES RELEASED] Gradescop |
RRR Week
| Apr 29: | ||
|---|---|---|
| Apr 30: | ||
| May 1: | ||
| May 2: | ||
| May 3: | [HOMEWORK 13 SELF-GRADES DUE] |
Finals Week
| Mar 6: | ||
|---|---|---|
| May 7: | ||
| May 8: | ||
| May 9: | [FINAL EXAM] Prob. PDF - Sol. PDF | |
| May 10: |
Convex Optimization
Convex Optimization book cover
Convex Optimization
Stephen Boyd and Lieven Vandenberghe
Cambridge University Press
A MOOC on convex optimization, CVX101, was run from 1/21/14 to 3/14/14. If you register for it, you can access all the course materials.
More material can be found at the web sites for EE364A (Stanford) or EE236B (UCLA), and our own web pages. Source code for almost all examples and figures in part 2 of the book is available in CVX (in the examples directory), in CVXOPT (in the book examples directory), and in CVXPY. Source code for examples in Chapters 9, 10, and 11 can be found here. Instructors can obtain complete solutions to exercises by email request to us; please give us the URL of the course you are teaching.
If you find an error not listed in our errata list, please do let us know about it.
Stephen Boyd & Lieven Vandenberghe
Download
Copyright in this book is held by Cambridge University Press, who have kindly agreed to allow us to keep the book available on the web.
-
Book
-
Lecture slides (updated summer 2023)
-
Original lecture slides
-
Additional exercises (migrated to github August 2022)
Catalog links
-
Cambridge Univ Press catalog entry
-
Amazon catalog entry
-
Tsinghua University Press Chinese translation
Projects
The following are the optional course projects this semester. Please note:
- The following projects are more or less final in terms of content. Feel free to start working on them. The problem set has not changed since the draft version.
- You must type your report in LaTeX. We provide a template below.
| Adversarial Machine Learning | Control of Multiplicative Noise Systems | Speeding Up Gradient Descent | |
|---|---|---|---|
| Problems | Prob. PDF - Prob. Code | Prob. PDF - Prob. Code | Prob. PDF - Prob. Code |
Report Template
You must type your report in LaTeX. This is to give you the chance to get a feel for what it’s like to write a technical research paper. You are recommended to use the following template: download here.
Resources
Officially Supported Resources
The following textbooks are highly recommended resources for the course.
- Calafiore, Giuseppe and El Ghaoui, Laurent. Optimization Models.
- Boyd, Stephen, and Vandenberghe, Lieven. Convex Optimization.
In addition, the staff from Spring 2023 wrote a course reader which follows the lectures; these notes have been augmented by every set of course staff since then.
Other Useful Resources
The following are resources that course staff thinks are helpful, but we do not want to officially support. They might be helpful to learn the material, but also might have bugs, etc. The list will expand periodically throughout the semester as we collect more resources.
- Petersen, Kaare and Pedersen, Michael. Matrix Cookbook.
- Bubeck, Sebastien. Convex Optimization: Algorithms and Complexity.
- Varaiya, Pravin. Lecture Notes on Optimization.
Past Exams
| Semester | Midterm/Midterm 1 | Midterm 2 | Final |
|---|---|---|---|
| Fall 2023 | Problems - Solutions | Problems - Solutions | |
| Spring 2023 | Problems - Solutions | Problems - Solutions | |
| Fall 2022 | Problems - Solutions | Problems - Solutions | |
| Spring 2020 | Problems - Solutions | Problems - Solutions | |
| Spring 2019 | Problems - Solutions | Problems - Solutions | Problems - Solutions |