On Thu, May 2, 2013 at 5:21 PM, Peter Prettenhofer <
peter.prettenho...@gmail.com> wrote:
> this looks pretty awesome - especially the dataset abstraction is pretty
> neat - would be great if we could merge this into scikit-learn.
>
Merging the dataset abstraction would be nice. We could port some of
scikit-learn's code to it, including SGD and mini-batch k-means. The neural
network PR by Lars could also benefit it.
BTW, do you think we should keep the weight vector abstraction which is in
scikit-learn?
> btw: what kind of truncated gradient algorithm does lightning use for L1
> penalized SGD? As far as I can see its not the one that's currently used in
> SGDClassifier...
>
It's the regular truncated SGD by Jonh Langford, which is identical to the
method described in the FOBOS paper. Compared to the one in scikit-learn,
it is more theoretically correct. The one in scikit-learn obtains sparser
weight vectors in practice but has no theoretical justification (it's an
heuristic). My goal was to compare coordinate descent with regular
truncated/projected SGD so I didn't implement this heuristic.
Mathieu
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
