Thanks for your approach, I didn't notice that cross_val_score accepts
cross validator as cv
Your approach makes that strange behavior disappeared!
But I still can't figure out what mistake I made, my original code looks
nothing wrong.
BTW, I used pipeline because I planned using data transformation.
2014-12-10 4:33 GMT+08:00 Sebastian Raschka <se.rasc...@gmail.com>:
> What is your dataset size? I am a little bit curious whether you need the
> pipe.fit(), I'd do the CV usually like this
>
> clf1 = Pipeline([
> ('classifier', RandomForestClassifier(n_estimators=100,
> min_samples_leaf=10,random_state=random.seed(1234)))
>
> cv = KFold(n=X_train.shape[0],
> n_folds=5,
> random_state=123)
>
> scores = cross_val_score(clf1, X_train, y_train, cv=cv, scoring='accuracy')
>
> Best,
> Sebastian
>
>
> > On Dec 9, 2014, at 3:05 PM, He-chien Tsai <depot...@gmail.com> wrote:
> >
> > I got two strange cross-validation scores even I tried different
> parameter of random_state in KFold, the last fold significantly lower than
> other folds like this:
> > [0.66555285540704734,
> > 0.64459295261239369,
> > 0.64611178614823817,
> > 0.6488456865127582,
> > 0.65268915223336377,
> > 0.65603160133697969,
> > 0.66423579459130966,
> > 0.097538742023700997]
> >
> > [0.82442284325637905,
> > 0.8353584447144593,
> > 0.82685297691373028,
> > 0.82320777642770349,
> > 0.82685297691373028,
> > 0.82989064398541923,
> > 0.82006079027355627,
> > 0.64133738601823709]
> > My code is below
> > pipe = Pipeline([
> > ('classifier', RandomForestClassifier(n_estimators=100,
> min_samples_leaf=10,random_state=random.seed(1234)))
> > ])
> > clfs = [ (pipe.fit(x[train_index], y[train_index]), (x[test_index],
> y[test_index])) for
> > train_index, test_index in KFold(x.shape[0], n_folds=8,
> shuffle=True, random_state=random.seed(125))]
> > scores = [m.accuracy_score(p[1][1], p[0].predict(p[1][0])) for p in clfs]
> >
> ------------------------------------------------------------------------------
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration & more
> > Get technology previously reserved for billion-dollar corporations, FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk_______________________________________________
> > Scikit-learn-general mailing list
> > Scikit-learn-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
