2012/7/5 Emanuele Olivetti <[email protected]>: > On 07/05/2012 09:45 AM, Andreas Mueller wrote: >> Hey Peter. >> Pretty awesome feat! Thanks for all the work you put into the ensemble >> module! >> >> A blog post about this competition would really be great :) >> >> I was wondering, was there much difference in performance between GBRT >> and RF? > > Hi, > > I participated to the competition (with not so good results ;-)) but > there was indeed a significant difference between RF and GB. I tried RF, > ExtraTrees > and GradientBoosting and got a RMSLE of ~0.60 using the first two (with > blending and a preprocessing procedure similar to the one of Peter) and > ~0.578 with GB. OK, you might say that the difference is little but there > are tens of positions in the final rank between them ;-)
This is amazing that the intrinsic variance of those is estimator is so small as to be able to say that a GBRT with mean loss around 0.578 beats RF with mean loss around 0.60 in a significant manner. What is the order of magnitude std error of 10 folds cross validation on this problem? 0.1? less? Is the ordering of the validation set and the final test set changed much? Did you observe wide differences between you internal CV score and the final test scores ? -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
