Thanks for this email. It is always nice to hear about success stories. I assume the guilty party is Issam Laradji, as you can see from his Google Summer of Code blog post:
http://issamlaradji.blogspot.jp/2014/06/week-3-gsoc-2014-extending-neural.html L-BFGS is indeed usually a good default choice for medium-scale datasets. It doesn't require any step size tuning and I found recently that it works well for poorly conditioned problems. You can also see a blog post by Nicolas Le Roux praising L-BFGS here: http://labs.criteo.com/2014/09/poh-part-3-distributed-optimization/ Mathieu On Sat, Aug 26, 2017 at 12:40 AM, Dr. Mario Michael Krell < mario.michael.kr...@gmail.com> wrote: > To whoever programmed the MLPClassifier (with the L-BFGS solver), > > I just wanted to personally thank you and if I get your name(s), I would > mention it/them in my paper additionally to the mandatory sklearn citation. > > I hope that sklearn will be keeping this algorithm forever in their > library despite the increasing amount of established deep learning > libraries that seem to make this code obsolete. For my small scale, more > theoretic analysis, it worked much better than any other algorithm and I > would not have gotten such surprising results. Due to the high quality > implementation, the integration of a much better solver than SGD, and the > respective good documentation, I could show empirically how the VC > dimension and another property of MLPs (MacKay dimension) actually scale > linear with the number of edges in the respective graph which helped us to > provide a new much more strict upper bound (https://arxiv.org/abs/1708. > 06019). This would have not been possible with other implementations. If > there is an interest by the developers, I could try to contribute a > tutorial documentation for sklearn. Just let me know. > > Thank you a lot!!! > > Best, > > Mario > > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn > >
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
