Hi,
I believe that this is a question for the scipy mailing list.
Gaël
On Tue, Jul 08, 2014 at 02:44:40PM +0200, Bao Thien wrote:
> Dear all,
> I need to optimize a loss function and currently use some optimizers from
> scipy.optimize.minimize
> More detail like this:
> + parameters to optimize : X - size is about 50
> + init parameters: X0
> + bounds - all parameters are in [0,1]
> + loss function: L (defined)
> + Jacobian (gradient) : J (defined)
> I have tried with many optimizers: SLSQP, L-BFGS-B, CG, Newton-CG, COBYLA.
> In the case that I don't provide the Jacobian, all the optimizers run and exit
> successfully with the final loss functions are more or less the same (the init
> loss is 21.18, and the final loss is ~16.87), but it takes sometime to finish.
> Then, I want to improve the speed of computation, I was advised to use
> Jacobian.
> However, when I provide the Jacobian there are something strange:
> - with SLSQP: the optimizer finish successfully, but the final loss is
> ~20.80
> (which is not much less then the init loss function value of 21.18)
> - with L-BFGS-B: ABNORMAL_TERMINATION_IN_LNSRCH
> Line search cannot locate an adequate point after 20 function
> and
> gradient evaluations. Previous x, f and g restored.
> Possible causes: 1 error in function or gradient evaluation;
> 2 rounding error dominate
> computation.t
> and returns exactly the init parameters: X0
> - with CG/Newton-CG: warning: Desired error not necessarily achieved due to
> precision loss.
> It returns the same loss function values as the init one: 21.18 and
> exactly the init parameters: X0.
> So, would anyone face this problem before, please let me know. Or if you have
> any hint or glue to overcome this problem, all are very appreciated.
> Regards,
> T.Bao
--
Gael Varoquaux
Researcher, INRIA Parietal
Laboratoire de Neuro-Imagerie Assistee par Ordinateur
NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France
Phone: ++ 33-1-69-08-79-68
http://gael-varoquaux.info http://twitter.com/GaelVaroquaux
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
