Sorry, my fault. Supervised PCA is different to Linear Discriminant Analysis. It uses a heuristic to keep only the variables that show some correlation with the response when calculating the components. It does not incorporate explicitly the class separation as an objective. Supervised PCA can be used for regression easily.
Regards, Stelios 2015-07-30 12:03 GMT+01:00 Mathieu Blondel <math...@mblondel.org>: > He was asking about Linear Discriminant Analysis, not Latent Dirichlet > Allocation. > > Mathieu > > On Thu, Jul 30, 2015 at 7:58 PM, Stylianos Kampakis < > stylianos.kampa...@gmail.com> wrote: > >> Hi Sebastian, >> >> LDA is unsupervised. Supervised PCA finds components correlated with the >> response variable. >> >> Best regards, >> Stelios >> >> 2015-07-29 22:55 GMT+01:00 Sebastian Raschka <se.rasc...@gmail.com>: >> >>> Out of curiosity, how does supervised PCA compare to LDA (Linear >>> Discriminant Analysis); in a nutshell, what would be the main difference? >>> >>> Best, >>> Sebastian >>> >>> On Jul 29, 2015, at 5:41 PM, Stylianos Kampakis < >>> stylianos.kampa...@gmail.com> wrote: >>> >>> Hi Andreas, >>> >>> Sure. Actually, the purpose of the model is both regularization and >>> dimensionality reduction for problems where the number of features can be >>> larger than the number of instances (or in any case when there is a large >>> number of features). It is particularly effective when there are lots of >>> highly correlated attributes with each other. >>> >>> L1 regularization breaks down in the presence of lots of correlations. >>> L2 deals better with this problem, but ignores the presence of clusters of >>> highly correlated attributes. Supervised PCA is particularly well suited to >>> these kinds of problems. The algorithm seems to outperform partial least >>> squares. >>> >>> I actually came up upon this algorithm when trying to find a way to >>> analyze GPS data gathered from the training of a professional football >>> team. Ridge logistic regression didn't provide good results, LASSO either, >>> but supervised PCA worked well. It is also possible to use it to reduce the >>> dimensionality in a way that the components correlate with the response. >>> >>> The work was presented at Mathsports International 2015 ( >>> http://www.mathsportinternational2015.com/uploads/2/2/2/4/22242920/mathsport2015proceedings.pdf >>> ) >>> >>> I am not sure about the popularity of this method, in general, but for >>> me it's going to be one of the standard methods to use in the presence of >>> lots of variables. >>> >>> Best regards, >>> Stelios >>> >>> 2015-07-28 19:16 GMT+01:00 Andreas Mueller <t3k...@gmail.com>: >>> >>>> Hi Stylianos. >>>> >>>> Can you give a bit more background on the model? >>>> It seems fairly well-cited but I haven't really seen it in practice. >>>> Is it still state of the art? >>>> The main purpose seems to be a particular type of regularization, >>>> right, not supervised dimensionality reduction? >>>> How does this compare against elastic net? There seems to be some >>>> comparison to PLS and lasso in the paper. >>>> >>>> It would be good to see that this is a widely useful method before >>>> adding it to sklearn. >>>> >>>> Cheers, >>>> Andy >>>> >>>> >>>> >>>> On 07/24/2015 06:40 AM, Stylianos Kampakis wrote: >>>> >>>> Dear all, >>>> >>>> I am thinking to contribute a new model to the library: The supervised >>>> principal components analysis by Bair et al. (2006). >>>> >>>> I wanted to get in touch before contributing to make sure no-one else >>>> is working on that algorithm, since this is what the site recommends. >>>> >>>> Cheers, >>>> S. Kampakis >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> >>>> _______________________________________________ >>>> Scikit-learn-general mailing >>>> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Scikit-learn-general mailing list >>>> Scikit-learn-general@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>>> >>>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> Scikit-learn-general@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Scikit-learn-general mailing list >>> Scikit-learn-general@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >>> >>> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
