On Feb 9, 2014, at 9:28 PM, Jun Kudo <[email protected]> wrote:
> What’s the reason for this behavior? From looking over Svanberg’s paper,
> the trust region, rho parameter, and the form of the MMA approximations seem
> to be independent of the magnitude of the function and gradients. I’d like
> to think that the initial steps should then be independent of the magnitude
> of the function/gradient.
>
Ideally, optimization algorithms would be invariant under rescaling of the
coordinates or of the function values, but unfortunately this is almost never
the case.
For example, equation (6.1a) of the Svanberg paper suggests initializing rho=1,
and NLopt uses this initial value. Unfortunately, rho is a dimensionful
constant (it has the same units as f), so if you rescale the objective function
then you are effectively changing the initial rho. This changes the initial
step sizes and hence the convergence rate: if rho is too large initially, then
the first few steps will be smaller than necessary.
Unfortunately, it seems quite difficult to get rid of these "magic"
dimensionful constants in most algorithms. The algorithm still eventually
converges, but they may affect the convergence rate, especially in the
beginning.
The upshot is that it is typically a good idea to scale your functions and
coordinates so that they have values of order unity, as that is the regime for
which the magic constants were probably tuned.
--SGJ
_______________________________________________
NLopt-discuss mailing list
[email protected]
http://ab-initio.mit.edu/cgi-bin/mailman/listinfo/nlopt-discuss
