Hi Amita,
As far as I understand your question, you only need one CV loop to optimize
your objective with scoring function provided:
===
pipeline=Pipeline([('scale', preprocessing.StandardScaler()),('filter',
SelectKBest(f_regression)),('svr', svm.SVR())]
C_range = [0.1, 1, 10, 100]
gamma_range=numpy.logspace(-2, 2, 5)
param_grid=[{'svr__kernel': ['rbf'], 'svr__gamma': gamma_range,'svr__C':
C_range}]
grid_search = GridSearchCV(pipeline, param_grid=param_grid, cv=5*,
scoring=scoring_function*)
grid_search.fit(X_train, Y_train)
===
More details about it you should be able to find in documentation:
- http://scikit-learn.org/stable/modules/grid_search.html#grid-search
-
http://scikit-learn.org/stable/modules/grid_search.html#gridsearch-scoring
2016-05-12 17:05 GMT+01:00 Amita Misra <amis...@ucsc.edu>:
> Hi,
>
> I have a limited dataset and hence want to learn the parameters and also
> evaluate the final model.
> From the documents it looks that nested cross validation is the way to do
> it. I have the code but still I want to be sure that I am not overfitting
> any way.
>
> pipeline=Pipeline([('scale', preprocessing.StandardScaler()),('filter',
> SelectKBest(f_regression)),('svr', svm.SVR())]
> C_range = [0.1, 1, 10, 100]
> gamma_range=numpy.logspace(-2, 2, 5)
> param_grid=[{'svr__kernel': ['rbf'], 'svr__gamma': gamma_range,'svr__C':
> C_range}]
> grid_search = GridSearchCV(pipeline, param_grid=param_grid,cv=5)
> Y_pred=cross_validation.cross_val_predict(grid_search, X_train,
> Y_train,cv=10)
>
> correlation= numpy.ma.corrcoef(Y_train,Y_pred)[0, 1]
>
>
> please let me know if my understanding is correct.
>
> This is 10*5 nested cross validation. Inner folds CV over training data
> involves a grid search over hyperparameters and outer folds evaluate the
> performance.
>
>
> Thanks,
> Amita--
> Amita Misra
> Graduate Student Researcher
> Natural Language and Dialogue Systems Lab
> Baskin School of Engineering
> University of California Santa Cruz
>
>
>
> ------------------------------------------------------------------------------
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Yours sincerely,
Alexey A. Dral
------------------------------------------------------------------------------
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
