Out of curiosity, how does supervised PCA compare to LDA (Linear Discriminant Analysis); in a nutshell, what would be the main difference?
Best, Sebastian > On Jul 29, 2015, at 5:41 PM, Stylianos Kampakis > <stylianos.kampa...@gmail.com> wrote: > > Hi Andreas, > > Sure. Actually, the purpose of the model is both regularization and > dimensionality reduction for problems where the number of features can be > larger than the number of instances (or in any case when there is a large > number of features). It is particularly effective when there are lots of > highly correlated attributes with each other. > > L1 regularization breaks down in the presence of lots of correlations. L2 > deals better with this problem, but ignores the presence of clusters of > highly correlated attributes. Supervised PCA is particularly well suited to > these kinds of problems. The algorithm seems to outperform partial least > squares. > > I actually came up upon this algorithm when trying to find a way to analyze > GPS data gathered from the training of a professional football team. Ridge > logistic regression didn't provide good results, LASSO either, but supervised > PCA worked well. It is also possible to use it to reduce the dimensionality > in a way that the components correlate with the response. > > The work was presented at Mathsports International 2015 > (http://www.mathsportinternational2015.com/uploads/2/2/2/4/22242920/mathsport2015proceedings.pdf > > <http://www.mathsportinternational2015.com/uploads/2/2/2/4/22242920/mathsport2015proceedings.pdf>) > > I am not sure about the popularity of this method, in general, but for me > it's going to be one of the standard methods to use in the presence of lots > of variables. > > Best regards, > Stelios > > 2015-07-28 19:16 GMT+01:00 Andreas Mueller <t3k...@gmail.com > <mailto:t3k...@gmail.com>>: > Hi Stylianos. > > Can you give a bit more background on the model? > It seems fairly well-cited but I haven't really seen it in practice. > Is it still state of the art? > The main purpose seems to be a particular type of regularization, right, not > supervised dimensionality reduction? > How does this compare against elastic net? There seems to be some comparison > to PLS and lasso in the paper. > > It would be good to see that this is a widely useful method before adding it > to sklearn. > > Cheers, > Andy > > > > On 07/24/2015 06:40 AM, Stylianos Kampakis wrote: >> Dear all, >> >> I am thinking to contribute a new model to the library: The supervised >> principal components analysis by Bair et al. (2006). >> >> I wanted to get in touch before contributing to make sure no-one else is >> working on that algorithm, since this is what the site recommends. >> >> Cheers, >> S. Kampakis >> >> >> ------------------------------------------------------------------------------ >> >> >> _______________________________________________ >> Scikit-learn-general mailing list >> Scikit-learn-general@lists.sourceforge.net >> <mailto:Scikit-learn-general@lists.sourceforge.net> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general >> <https://lists.sourceforge.net/lists/listinfo/scikit-learn-general> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > <mailto:Scikit-learn-general@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > <https://lists.sourceforge.net/lists/listinfo/scikit-learn-general> > > > ------------------------------------------------------------------------------ > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
