hi gael,
If you are using a StratifiedKFold, it seems to me that you are in
> classification settings. If the error metric that you are using is the
> default 0-1 loss, i.e. the mean of the prediction errors, than the two
> options that you are refering to are very similar. If the folds have the
> same size, than they are exactly mathematically equal.
>
yes in certain settings and depending on the score func, they will be
identical. i'm using avg_f1_score (based on andreas' email - not yet pushed
yet) and the results are relatively close between the two methods, but not
identical.
> I don't really see how a different averaging strategy across folds would
> improve the variance in these settings, but maybe I am missing something?
>
the variance statement was only in relation to selection of 'k' and how
that relates test set size relative to the training set (all numbers are
small over here, so variance fluctuates quite a bit). i didn't mean it in
relation to the averaging across folds.
cheers,
satra
------------------------------------------------------------------------------
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
