SYSTEMATIC MATHEMATICS

Optimization & Convex Analysis

This course develops forty-two visible chapters in eight content-sized units of lengths 5, 5, 6, 6, 5, 6, 5, and 4. It begins by separating a decision variable, feasible set, objective, units, and data source, then proves when a minimum exists before any algorithm is trusted. Smooth local conditions lead to Hessian and Newton models; convex sets and functions then turn supporting planes, subgradients, and conjugates into global certificates. Equality and inequality constraints are handled through tangent spaces, KKT conditions, constraint qualifications, weak and strong duality, and sensitivity. The algorithm half derives gradient, line-search, momentum, projected, proximal, coordinate, stochastic, subgradient, ADMM, simplex, quadratic, conic, and branch-and-bound reasoning with explicit residuals and failure boundaries. The final dossier reconstructs one constrained allocation from model through exact certificate and perturbation. The course does not claim to replace infinite-dimensional variational analysis, specialized integer or global optimization, stochastic approximation theory, or solver-specific large-scale numerical training.

Before this course: Completed Multivariable Calculus, Linear Algebra, and Real Analysis. The course assumes vectors, norms, affine sets, multivariable derivatives, Taylor expansion, symmetric matrices, eigenvalues, compactness, continuity, sequences, and proof. It does not assume a commercial solver, programming language, probability beyond finite averages, functional analysis, or measure theory.

COURSE FACTSLevel, chapters, units, prerequisite, and outcome
Chapter 1

Variables, objectives, and feasible sets define the problem

Objective: Why is x=3 not an acceptable answer even though f′(3)=0?

Optimization compares permitted decisions. The same formula with a different feasible set is a different problem, and a technically small objective value is irrelevant if the decision violates a constraint. Start by naming the decision variable, its allowed set, the quantity being minimized or maximized, and the units of every coefficient. The chapter’s exact result is: Adding a constant to an objective changes every objective value by the same amount and therefore leaves the set of minimizers unchanged.

The precise object is: An optimization problem has a decision variable x in a feasible set C and an objective f; “minimize f(x) subject to x∈C” asks for x*∈C with f(x*)≤f(x) for every x∈C. Read “minimize” as a comparison over every feasible choice, not as an instruction to differentiate immediately. A derivative, multiplier, or algorithm is useful only after its hypotheses have been checked.

The concrete problem is: Minimize f(x)=(x−3)²+1 subject to 0≤x≤2. Reconstruct the three worked steps, verify the reported value independently, and then test this nearby failure boundary: Changing the feasible set can change the answer even when the objective formula is untouched; feasibility is not a side note.

Adding a constant to an objective changes every objective value by the same amount and therefore leaves the set of minimizers unchanged.

For any feasible x and y, f(x)≤f(y) exactly when f(x)+c≤f(y)+c.

Thus every pairwise ordering of feasible decisions is preserved.

A point minimizes f over C exactly when it minimizes f+c over C.

Minimize f(x)=(x−3)²+1 subject to 0≤x≤2.

  1. The unconstrained vertex is x=3, but it is not feasible because 3>2.
  2. On [0,2], the distance |x−3| decreases as x moves toward the right endpoint.
  3. Choose x*=2 and compute f(2)=1²+1=2; every smaller feasible x is farther from 3.

Result: The unique feasible minimizer is x*=2 and the minimum value is 2.