# Optimization

**Type:** Concept  
**Domain:** Applied Mathematics  
**Codex URL:** /mathematics/applied/optimization/  
**Entry status:** Live — v1.0 (2026-05-27)

## Summary
Finds the best solution from feasible options. Underlies machine learning, logistics, engineering design.

## Unconstrained optimization
Critical points where f'(x)=0; second derivative test (or Hessian matrix for multivariable) determines max/min.

## Lagrange multipliers (1788)
For constrained optimization: ∇f=λ∇g plus the constraint itself. λ has an economic "shadow price" interpretation.

## Linear programming
Dantzig's simplex method (1947) — moves between corners of the feasible region, improving objective each step.

## Gradient descent
Iterative: step in negative gradient direction. Combined with backpropagation, trains virtually all modern neural networks.

## Convex optimization
Local minimum = global minimum for convex functions — hugely valuable property.

## Calculus of variations
Optimizes over entire functions/curves, not just points. Brachistochrone problem (Johann Bernoulli, 1696) — classic example, connects to Principle of Least Action in physics.

## Combinatorial optimization / NP-hardness
Many discrete optimization problems (e.g. Travelling Salesman) are NP-hard — motivates heuristic/approximation algorithms.

## Sources
### Tier 1
- Boyd, S. and Vandenberghe, L. (2004). *Convex Optimization*.
- Nocedal, J. and Wright, S.J. (2006). *Numerical Optimization*. 2nd ed.
### Tier 2
- Dantzig, G.B. (1963). *Linear Programming and Extensions*.

---
*Mathematics Codex entry v1.0 — added 2026-05-27 — thecodex.expert/mathematics/*