I am not sure that K-SVD is any better, but one of the other people in my
group proposed using K-SVD, so I will likely have to implement it. It will
be worth a comparison - I will try and stick to the sklearn API.
On Dec 13, 2013 2:02 PM, "Vlad Niculae" <zephy...@gmail.com> wrote:
> We did not implement K-SVD because we did not find any motivation for
> having two competing dictionary learning implementations, so we stuck
> with the Julien Mairal et al solver. Do you think that K-SVD would do
> better than it for this?
>
> Vlad
>
>
> On Fri, Dec 13, 2013 at 8:46 PM, Kyle Kastner <kastnerk...@gmail.com>
> wrote:
> > I have 2 separate approaches I am considering for real-world testing.
> >
> > For kaggle cats and dogs, using a deep neural network trained on ImageNet
> > (DeCAF http://arxiv.org/abs/1310.1531) for preprocessing, coupled with
> any
> > kind of classifier, has had excellent success for me so far (even
> logistic
> > regression was ~95% accurate). The best result I have so far is DeCAF
> > preprocessing, followed by a 4 layer deep neural net. I am not really
> doing
> > anything special with the classifier - the discrimination power appears
> to
> > be primarily in the features output from the DeCAF network. It could be
> > interesting to try and reimplement/wrap the pretrained network in sklearn
> > somehow... though the authors have a newer framework called Caffe now
> > http://daggerfs.com/caffe/
> >
> > I am thinking of using KMeansCoder features as a comparison - my guess is
> > that it will not be as good (or at least shouldn't be!), but for an
> > incredible reduction in complexity the tradeoff may be worth it in other
> > applications, where a dataset like ImageNet is not available. My primary
> > dataset is speech/communications signals, and I am trying to use these
> > techniques for cognitive radio/spectral sensing.
> >
> > Eventually, a stacked KMeans approach will be evaluated - basically
> multiple
> > layers of KMeans coders, as in 'Learning Feature Representations with
> > K-means' by A. Coates and A. Ng. My primary dataset is unsupervised, so
> the
> > "learn a huge neural net and use it as pre-processing" technique will
> > probably not work, unless there is a big labeled dataset somewhere else I
> > haven't seen.
> >
> > I will report back when there are some "real world" results - either for
> > speech/comms or dogs/cats. Thanks for writing this code originally! It
> is a
> > testament to the project that code from two years ago can be brought to a
> > working state with ~5 lines of minor modifications. I am also planning to
> > evaluate a K-SVD dictionary learning approach - does anyone know if that
> is
> > currently implemented/in development for sklearn? I haven't looked for
> it in
> > sklearn yet, but it seems like a cool approach
> >
> >
> > On Fri, Dec 13, 2013 at 12:20 PM, Vlad Niculae <zephy...@gmail.com>
> wrote:
> >>
> >> Great, thanks a lot!
> >>
> >> I'm also curious about what you're running it on and about how the
> >> performance is.
> >>
> >> Vlad
> >>
> >> On Fri, Dec 13, 2013 at 7:11 PM, Olivier Grisel
> >> <olivier.gri...@ensta.org> wrote:
> >> > Nice.
> >> >
> >> > Have you used it with success for real image classification tasks?
> >> >
> >> > I see you have been involved in the cats vs dogs kaggle competition.
> >> > Is learning a linear model, if so we might consider including the such
> >> > KMeansCoder as part of the sklearn.feature_extraction.image module and
> >> > write an example for that dataset.
> >> >
> >> > Many people ask us how to use scikit-learn for image classification
> >> > and we have no getting started example to point them at. If the KMeans
> >> > patch encoder proves to be a reasonable baseline I would be +1 for
> >> > having it as part of scikit-learn.
> >> >
> >> > Do you do some max pooling + normalization on the output?
> >> >
> >> > --
> >> > Olivier
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > Rapidly troubleshoot problems before they affect your business. Most
> IT
> >> > organizations don't have a clear picture of how application
> performance
> >> > affects their revenue. With AppDynamics, you get 100% visibility into
> >> > your
> >> > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
> >> > AppDynamics Pro!
> >> >
> >> >
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> >> > _______________________________________________
> >> > Scikit-learn-general mailing list
> >> > Scikit-learn-general@lists.sourceforge.net
> >> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Rapidly troubleshoot problems before they affect your business. Most IT
> >> organizations don't have a clear picture of how application performance
> >> affects their revenue. With AppDynamics, you get 100% visibility into
> your
> >> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
> AppDynamics
> >> Pro!
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Scikit-learn-general mailing list
> >> Scikit-learn-general@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Rapidly troubleshoot problems before they affect your business. Most IT
> > organizations don't have a clear picture of how application performance
> > affects their revenue. With AppDynamics, you get 100% visibility into
> your
> > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> > Pro!
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Scikit-learn-general mailing list
> > Scikit-learn-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
