8.3
In constrained minimization, extra constraints, either equalities or inequalities, are imposed on the solution f. The problem with K equality constraints is stated as
where 
denotes the equality constraints. For an inequality
constraint of the form 
, we can introduce a nonnegative slack
function of one variable, e.g.
, and convert it into an equality
constraint 
. So only problems with equality constraints are
considered here. The constraints define the feasible region 
for
f.
The problem is termed linear programming when the energy function and
the constraint functions are all linear in f; otherwise it is
nonlinear. Methods for linear and nonlinear programming have quite
different features. Because our energy functions 
can be nonlinear
in f, our discussion will be on nonlinear programming problems.
When the constraint functions are linear, the gradient projection method
[Rosen 1960] can be used to find feasible directions for the steepest
descent. Given that 
is in the feasible region, the gradient
is computed. Direct gradient descent gives 
. But this vector may no longer be in 
. To keep the
feasibility, the new configuration is obtained by projecting
onto 
. By this, the updated configuration
still lies in the feasible region. The computation is not difficult when
the constraints are linear such that 
corresponds to a
hyperplane. This is the GP operator used in RL described earlier.
In classical methods for nonlinear programming, constrained minimization problems are converted into unconstrained ones, e.g. by using penalty functions and Lagrange multipliers. Unlike in RL where the feasibility constraints are respected throughout the update process, they may not necessarily be so for penalty and Lagrange methods.