Dear Steven,
Thank you for investigating.
One thing that would be helpful would be if someone tried Alexander's
test problem in fmin_slsqp from SciPy, which also uses the SLSQP code,
to see if a similar problem (albeit probably with different symptoms)
occurs there.
Unfortunately, I cannot help you with that. I have no idea about Python.
Tracing through the code, this appears to be a problem in the original
SLSQP software. SLSQP works by minimizing an approximate quadratic
subproblem at each step. At the first step of Alexander's 1d test case,
SLSQP is minimizing:
min_x (x - s)^2
-1 <= x <= 1
I wonder whether this is really the problem actually solved by the
method. I must admit, I have not checked the code, and I am not sure
about SLSQP. The original problem is
min_x f(x) = -s * x
g(x) = (x-1, -x-1)^T <= 0.
In a general SQP method, one would then solve
min_d -s * d
(x_k - 1, -x_k - 1) + (d, -d) <=0,
where x_k is the previous iterate, and then set
x_{k+1} = x_k + d.
(Actually, one would perform a line search with Armijo rule and search
direction d.)
There is no quadratic term in the target function in this case, since
the second derivative of the Lagrange function
L(x) = f(x) + lambda^T g(x) = -s*x + lambda^T (x-1, -x-1)^T
is 0.
Perhaps SLSQP somehow transforms this problem, which is quadratic in
general, to the minimization problem you state. I probably should have a
look at the original papers by Dieter Kraft describing SLSQP.
Unfortunately, the guts of the LSQ routine in SLSQP are rather painful
to debug; this is pretty ugly spaghetti-style code (even before f2c
conversion from Fortran to C),
That is very true ;-).
For now, your options seem to be (a) rescale your problem so that
gradients etcetera are of order 1 or (b) use a different algorithm in
NLopt.
My actual application is highly non-linear, 50-100 dimensional, and
unfortunately too complex to do any rescaling. While LBFGS works, it is
not really an option, since it needs too many gradient evaluations. SQP
is the only chance to get results within reasonable times (less than 12
hours).
Knowing that there may be numerical instabilities in the from my point
of view most important optimization algorithm offered by NLOPT is
somewhat scaring...
Best regards,
Peter
_______________________________________________
NLopt-discuss mailing list
[email protected]
http://ab-initio.mit.edu/cgi-bin/mailman/listinfo/nlopt-discuss
