Hi, I am writing a fault detection system using OCSVM. I start with a huge matrix shape=(1006, 300000) and I reduce its dimension with IncrementalPCA.
If I use 10 - 100 pca components I get a very good AUC score around 0.97 while with 1000 components it drops to 0.5. Is it possible that incremental PCA becomes unstable when too many components are used? I can not find another explanation for the drop in performances. As far as I know OCSVM should be able to scale well to high dimensional datasets. The code is here http://jpst.it/CpqB Let me know. Thanks! -- Sent by mobile phone
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
