Apropros Mahalanobis distance: To get back to the initial question about scale invariant classifiers ...
As Andreas already pointed out: The tree-based methods are scale invariant. But under certain circumstances, you can also add Nearest Neighbor classifiers and kernel methods such as kernel SVM to the list depending on which distance metric you choose. But note that by default, the KNN in scikit-learn uses Euclidean distance (you can change it to "mahalanobis" though), and the RBF kernel in scikit-learns SVM also uses an Euclidean distance measure, so those are not scale invariant. > On Jun 4, 2015, at 2:38 PM, Sturla Molden <sturla.mol...@gmail.com> wrote: > > On 04/06/15 20:18, Andreas Mueller wrote: > >> I'm not sure what you mean by that. The cluster-memberships? The means >> and covariances will certainly be different. > > The Mahalanobis distance will undo any linear scaling operation. > >> They are actually somewhat regularized in scikit-learn, by having a >> minimum diagonal covariance. > > Good. > > Component-wise EM (aka CEM2) is a better way of avoiding the singularity > disease, though. > > > Sturla > > > > ------------------------------------------------------------------------------ > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
