Changing the cv parameter (number of iterations) in cross_val_score() really changes the returned scores. Increasing the CV doesn't neccessarily mean that the returned scores stabalise. Instead, they get worse, and only get better later. I have included the output of increasing the CV below.
My questions are: 1. How is it possible that the accuracy of a prediction is sometimes negative? Isn't the lowest accuracy 0.00? 2. The default value of CV (if none is specified is 3). However, when there is so much varience in the output of different CV values, how do you know which one is accurate? Which one should you choose? 3. Why do the accuracy and std become so variable as the CV increases, maax out around the middle (CV 21), and only then stabalise again ? 53 cases, 6596 features: CV: 2 Accuracy: 0.40 (+/- 0.00) CV: 3 Accuracy: 0.33 (+/- 0.10) CV: 4 Accuracy: 0.45 (+/- 0.04) CV: 5 Accuracy: 0.40 (+/- 0.05) CV: 6 Accuracy: 0.25 (+/- 0.19) CV: 7 Accuracy: -3.55 (+/- 4.77) CV: 8 Accuracy: 0.26 (+/- 0.17) CV: 9 Accuracy: 0.27 (+/- 0.21) CV: 10 Accuracy: -6.12 (+/- 9.69) CV: 11 Accuracy: -2.48 (+/- 4.52) CV: 12 Accuracy: -2.28 (+/- 4.34) CV: 13 Accuracy: -7.13 (+/- 9.51) CV: 14 Accuracy: -41.77 (+/- 71.35) CV: 15 Accuracy: -38.97 (+/- 69.13) CV: 16 Accuracy: -40.49 (+/- 67.01) CV: 17 Accuracy: -48.56 (+/- 66.93) CV: 18 Accuracy: -98.55 (+/- 96.74) CV: 19 Accuracy: -99.74 (+/- 94.20) CV: 20 Accuracy: -95.06 (+/- 92.38) CV: 21 Accuracy: -100.46 (+/- 90.95) CV: 22 Accuracy: -95.88 (+/- 89.48) CV: 23 Accuracy: -91.69 (+/- 88.06) CV: 24 Accuracy: -90.05 (+/- 86.31) CV: 25 Accuracy: -93.43 (+/- 84.81) CV: 26 Accuracy: -95.33 (+/- 83.39) CV: 27 Accuracy: 0.01 (+/- 0.04) CV: 28 Accuracy: 0.01 (+/- 0.04) CV: 29 Accuracy: 0.01 (+/- 0.03) CV: 30 Accuracy: 0.01 (+/- 0.03) CV: 31 Accuracy: 0.01 (+/- 0.03) CV: 32 Accuracy: 0.01 (+/- 0.03) CV: 33 Accuracy: 0.01 (+/- 0.02) CV: 34 Accuracy: 0.01 (+/- 0.02) CV: 35 Accuracy: 0.00 (+/- 0.01) CV: 36 Accuracy: 0.00 (+/- 0.01) CV: 37 Accuracy: 0.01 (+/- 0.02) CV: 38 Accuracy: 0.01 (+/- 0.02) CV: 39 Accuracy: 0.01 (+/- 0.03) CV: 40 Accuracy: 0.01 (+/- 0.03) CV: 41 Accuracy: 0.01 (+/- 0.02) CV: 42 Accuracy: 0.01 (+/- 0.02) CV: 43 Accuracy: -0.00 (+/- 0.01) CV: 44 Accuracy: -0.02 (+/- 0.05) CV: 45 Accuracy: -0.01 (+/- 0.05) CV: 46 Accuracy: -1.40 (+/- 4.68) CV: 47 Accuracy: -1.14 (+/- 3.87) CV: 48 Accuracy: -0.97 (+/- 3.31) CV: 49 Accuracy: -1.35 (+/- 4.69) CV: 50 Accuracy: -1.11 (+/- 3.90) CV: 51 Accuracy: -1.11 (+/- 3.93) CV: 52 Accuracy: 0.00 (+/- 0.00) CV: 53 Accuracy: 0.00 (+/- 0.00) Zach ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
