Vlad, if you are going to cite the Mazack paper, you should also look into
the original paper by Kim and Park (2006) which proposes the sparse NMF
model ( the one which is currently implemented by Scikit-learn).
For the sparse NMF model by Hoyer (2004), some benchmarks can be found in
my ICLR paper.
They were written in Matlab. Will convert them to Python as well and send
you the link.
Thanks,
~Vamsi.
On Wed, Jun 25, 2014 at 11:37 AM, Vlad Niculae <zephy...@gmail.com> wrote:
> Vamsi, do you have any benchmarks for your implementation? The biggest
> reason why we didn't change the current implementation yet is that it
> was hard to find something else that is consistently faster & better.
>
> Here are some benchmarks I did last time I looked at this, when I
> wanted to replace the hacky regularization currently implemented with
> an actual elastic net penalty. [1] is for just the non-negative least
> squares solver, [2] and [3] are NMF implementations on top of the PG
> and multiplicative methods.
> I was pretty disappointed with not managing to find one configuration
> that is consistently better, so I stopped pursuing this, as I'm not
> using NMF in my work at the moment.
>
> Vlad
>
> [1]
> http://nbviewer.ipython.org/urls/gist.githubusercontent.com/vene/7224672/raw/f9ab27a74addcca9e5e911c47478641d137ebc35/nls_solvers.ipynb
> [2]
> http://nbviewer.ipython.org/urls/gist.githubusercontent.com/vene/7196622/raw/7c7764ca9d0a6effba4976c5818933ef058a01ac/compare_nmf.ipynb
> [3]
> http://nbviewer.ipython.org/urls/gist.github.com/vene/7196622/raw/63f0005cf37cccf6d3b58b3d63b8aa9f6701b36b/compare_nmf_faces.ipynb
>
> On Wed, Jun 25, 2014 at 11:28 AM, Michael Eickenberg
> <michael.eickenb...@gmail.com> wrote:
> > i will never post a docstring again :)
> > sorry for the noise
> >
> > michael
> >
> >
> > On Wednesday, June 25, 2014, Vlad Niculae <zephy...@gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> Allow me to clarify. We don't implement Hoyer's sparse update rule
> >> indeed (it shouldn't say "this implements", I initially cited Hoyer
> >> for motivating sparseness constraints in NMF). Instead, we implement a
> >> version of sparse NMF with a clear (but not particularly elegant)
> >> objective function, found in [1].
> >>
> >> I had somewhere an implementation that just alternatively optimizes
> >> elastic net problems in both W and H, which is more elegant but I
> >> didn't finish polishing and benchmarking it.
> >>
> >> I should probably update the docstring, sorry for the confusion.
> >>
> >> Yours,
> >> Vlad
> >>
> >> [1] http://mazack.org/papers/mazack_nmf_paper.pdf
> >>
> >> On Wed, Jun 25, 2014 at 11:00 AM, Vamsi Krishna Potluru
> >> <vamsi.potl...@rutgers.edu> wrote:
> >> > Don't believe all you read :-).
> >> > It says that but implements something else.
> >> >
> >> > ~Vamsi.
> >> >
> >> >
> >> > On Wed, Jun 25, 2014 at 10:48 AM, Michael Eickenberg
> >> > <michael.eickenb...@gmail.com> wrote:
> >> >>
> >> >>
> >> >>
> >> >>
> https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/decomposition/nmf.py#L346
> >> >>
> >> >>
> >> >>
> >> >> References
> >> >>
> >> >> ----------
> >> >> This implements
> >> >>
> >> >>
> >> >> C.-J. Lin. Projected gradient methods
> >> >> for non-negative matrix factorization. Neural
> >> >> Computation, 19(2007), 2756-2779.
> >> >> http://www.csie.ntu.edu.tw/~cjlin/nmf/
> >> >>
> >> >>
> >> >> P. Hoyer. Non-negative Matrix Factorization with
> >> >> Sparseness Constraints. Journal of Machine Learning
> >> >> Research 2004.
> >> >>
> >> >>
> >> >>
> >> >> On Wed, Jun 25, 2014 at 4:39 PM, Vamsi Krishna Potluru
> >> >> <vamsi.potl...@rutgers.edu> wrote:
> >> >>>
> >> >>> Yes, thats a different sparse model than the one in Hoyer, 2004
> (most
> >> >>> likely the one by Kim and Park).
> >> >>> Maybe, Vlad or someone else can comment on that.
> >> >>>
> >> >>> ~Vamsi.
> >> >>>
> >> >>>
> >> >>> On Wed, Jun 25, 2014 at 10:15 AM, Alexandre Gramfort
> >> >>> <alexandre.gramf...@telecom-paristech.fr> wrote:
> >> >>>>
> >> >>>> hi,
> >> >>>>
> >> >>>> have you played with the sparseness parameter of the NMF estimator?
> >> >>>>
> >> >>>> Alex
> >> >>>>
> >> >>>>
> >> >>>> On Wed, Jun 25, 2014 at 3:42 PM, Vamsi Krishna Potluru
> >> >>>> <vamsi.potl...@rutgers.edu> wrote:
> >> >>>> > Hello,
> >> >>>> >
> >> >>>> > I have worked a bit on the sparse NMF model proposed by Hoyer
> [1].
> >> >>>> > The
> >> >>>> > paper is mentioned in the Scikits NMF module but AFAIK the model
> is
> >> >>>> > currently not implemented. Recently, we proposed an efficient
> >> >>>> > algorithm
> >> >>>> > based on block coordinate descent [2]. A reference python
> >> >>>> > implementation is
> >> >>>> > available at:
> >> >>>> >
> >> >>>> > https://github.com/ismav/sparseNMF
> >> >>>> >
> >> >>>> > Kindly let me know if this would be of interest to the
> >> >>>> > Scikits-learn
> >> >>>> > community.
> >> >>>> >
> >> >>>> > Thanks,
> >> >>>> >
> >> >>>> > ~Vamsi.
> >> >>>> >
> >> >>>> >
> >> >>>> > [1] Hoyer, P. O. (2004). Non-negative Matrix Factorization with
> >> >>>> > Sparseness
> >> >>>> > Constraints. Journal of Machine Learning Research, 5, 1457-1469.
> >> >>>> >
> >> >>>> > [2] Block Coordinate Descent for Sparse NMF
> >> >>>> >
> >> >>>> > Vamsi K. Potluru, Sergey M. Plis, Jonathan Le Roux, Barak A.
> >> >>>> > Pearlmutter,
> >> >>>> > Vince D. Calhoun, Thomas P. Hayes. ICLR 2013.
> >> >>>> >
> >> >>>> > http://arxiv.org/abs/1301.3527
> >> >>>> >
> >> >>>> >
> >> >>>> >
> >> >>>> >
> >> >>>> >
> ------------------------------------------------------------------------------
> >> >>>> > Open source business process management suite built on Java and
> >> >>>> > Eclipse
> >> >>>> > Turn processes into business applications with Bonita BPM
> Community
> >> >>>> > Edition
> >> >>>> > Quickly connect people, data, and systems into organized
> workflows
> >> >>>> > Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> >>>> > http://p.sf.net/sfu/Bonitasoft
> >> >>>> > _______________________________________________
> >> >>>> > Scikit-learn-general mailing list
> >> >>>> > Scikit-learn-general@lists.sourceforge.net
> >> >>>> >
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >> >>>> >
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> ------------------------------------------------------------------------------
> >> >>>> Open source business process management suite built on Java and
> >> >>>> Eclipse
> >> >>>> Turn processes into business applications with Bonita BPM Community
> >> >>>> Edition
> >> >>>> Quickly connect people, data, and systems into organized workflows
> >> >>>> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> >>>> http://p.sf.net/sfu/Bonitasoft
> >> >>>> _______________________________________________
> >> >>>> Scikit-learn-general mailing list
> >> >>>> Scikit-learn-general@lists.sourceforge.net
> >> >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> ------------------------------------------------------------------------------
> >> >>> Open source business process management suite built on Java and
> >> >>> Eclipse
> >> >>> Turn processes into business applications with Bonita BPM Community
> >> >>> Edition
> >> >>> Quickly connect people, data, and systems into organized workflows
> >> >>> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> >>> http://p.sf.net/sfu/Bonitasoft
> >> >>> _______________________________________________
> >> >>> Scikit-learn-general mailing list
> >> >>> Scikit-learn-general@lists.sourceforge.net
> >> >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> Open source business process management suite built on Java and
> Eclipse
> >> >> Turn processes into business applications with Bonita BPM Community
> >> >> Edition
> >> >> Quickly connect people, data, and systems into organized workflows
> >> >> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> >> http://p.sf.net/sfu/Bonitasoft
> >> >> _______________________________________________
> >> >> Scikit-learn-general mailing list
> >> >> Scikit-learn-general@lists.sourceforge.net
> >> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >> >>
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Open source business process management suite built on Java and
> Eclipse
> >> > Turn processes into business applications with Bonita BPM Community
> >> > Edition
> >> > Quickly connect people, data, and systems into organized workflows
> >> > Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> > http://p.sf.net/sfu/Bonitasoft
> >> > _______________________________________________
> >> > Scikit-learn-general mailing list
> >> > Scikit-learn-general@lists.sourceforge.net
> >> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Open source business process management suite built on Java and Eclipse
> >> Turn processes into business applications with Bonita BPM Community
> >> Edition
> >> Quickly connect people, data, and systems into organized workflows
> >> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> >> http://p.sf.net/sfu/Bonitasoft
> >> _______________________________________________
> >> Scikit-learn-general mailing list
> >> Scikit-learn-general@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >
> >
> >
> ------------------------------------------------------------------------------
> > Open source business process management suite built on Java and Eclipse
> > Turn processes into business applications with Bonita BPM Community
> Edition
> > Quickly connect people, data, and systems into organized workflows
> > Winner of BOSSIE, CODIE, OW2 and Gartner awards
> > http://p.sf.net/sfu/Bonitasoft
> > _______________________________________________
> > Scikit-learn-general mailing list
> > Scikit-learn-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >
>
>
> ------------------------------------------------------------------------------
> Open source business process management suite built on Java and Eclipse
> Turn processes into business applications with Bonita BPM Community Edition
> Quickly connect people, data, and systems into organized workflows
> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> http://p.sf.net/sfu/Bonitasoft
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
