Hi Andy, > > I get different order of importance for my variables depending on their order in the training data.
Perhaps answering my own question, the change in importance rankings could be attributed to the fact that before passing my data to randomForest I impute the missing values randomly (using the combined distributions of pos+neg), so the data seen by RF is slightly different. Then combining this with the fact that RF chooses data randomly it makes sense to see different rankings. In a previous thread regarding simplifying variables: http://thread.gmane.org/gmane.comp.lang.r.general/6989/focus=6993 you say: "The basic problem is that when you select important variables by RF and then re-run RF with those variables, the OOB error rate become biased downward. As you iterate more times, the "overfitting" becomes more and more severe (in the sense that, the OOB error rate will keep decreasing while error rate on an independent test set will be flat or increases)" But if every time you remove a variable you pass some test data (ie data not used to train the model) and base the performance of the new, reduced model on the error rate on the confusion matrix for the test data, then this "overfitting" should not be an issue, right? (unless of course you were referring to unsupervised learning). Best regards Eleni Rapsomaniki Birkbeck College, UK ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
