He was asking about Linear Discriminant Analysis, not Latent Dirichlet Allocation.
Mathieu On Thu, Jul 30, 2015 at 7:58 PM, Stylianos Kampakis < stylianos.kampa...@gmail.com> wrote: > Hi Sebastian, > > LDA is unsupervised. Supervised PCA finds components correlated with the > response variable. > > Best regards, > Stelios > > 2015-07-29 22:55 GMT+01:00 Sebastian Raschka <se.rasc...@gmail.com>: > >> Out of curiosity, how does supervised PCA compare to LDA (Linear >> Discriminant Analysis); in a nutshell, what would be the main difference? >> >> Best, >> Sebastian >> >> On Jul 29, 2015, at 5:41 PM, Stylianos Kampakis < >> stylianos.kampa...@gmail.com> wrote: >> >> Hi Andreas, >> >> Sure. Actually, the purpose of the model is both regularization and >> dimensionality reduction for problems where the number of features can be >> larger than the number of instances (or in any case when there is a large >> number of features). It is particularly effective when there are lots of >> highly correlated attributes with each other. >> >> L1 regularization breaks down in the presence of lots of correlations. L2 >> deals better with this problem, but ignores the presence of clusters of >> highly correlated attributes. Supervised PCA is particularly well suited to >> these kinds of problems. The algorithm seems to outperform partial least >> squares. >> >> I actually came up upon this algorithm when trying to find a way to >> analyze GPS data gathered from the training of a professional football >> team. Ridge logistic regression didn't provide good results, LASSO either, >> but supervised PCA worked well. It is also possible to use it to reduce the >> dimensionality in a way that the components correlate with the response. >> >> The work was presented at Mathsports International 2015 ( >> http://www.mathsportinternational2015.com/uploads/2/2/2/4/22242920/mathsport2015proceedings.pdf >> ) >> >> I am not sure about the popularity of this method, in general, but for me >> it's going to be one of the standard methods to use in the presence of lots >> of variables. >> >> Best regards, >> Stelios >> >> 2015-07-28 19:16 GMT+01:00 Andreas Mueller <t3k...@gmail.com>: >> >>> Hi Stylianos. >>> >>> Can you give a bit more background on the model? >>> It seems fairly well-cited but I haven't really seen it in practice. >>> Is it still state of the art? >>> The main purpose seems to be a particular type of regularization, right, >>> not supervised dimensionality reduction? >>> How does this compare against elastic net? There seems to be some >>> comparison to PLS and lasso in the paper. >>> >>> It would be good to see that this is a widely useful method before >>> adding it to sklearn. >>> >>> Cheers, >>> Andy >>> >>> >>> >>> On 07/24/2015 06:40 AM, Stylianos Kampakis wrote: >>> >>> Dear all, >>> >>> I am thinking to contribute a new model to the library: The supervised >>> principal components analysis by Bair et al. (2006). >>> >>> I wanted to get in touch before contributing to make sure no-one else >>> is working on that algorithm, since this is what the site recommends. >>> >>> Cheers, >>> S. Kampakis >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> >>> _______________________________________________ >>> Scikit-learn-general mailing >>> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> Scikit-learn-general@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
