Hi Joel,
Unfortunately, the link says "artifact not found". Whatever that means..
On 29/11/16 09:50, Joel Nothman wrote:
This makes me a little sad. Do Albert and Daniel think the explicit
reference from blurb to code proposed at
https://github.com/scikit-learn/scikit-learn/pull/7949 is a sufficient
remedy? Otherwise could you please propose another clarifying change?
Thanks.
On 29 November 2016 at 20:04, Albert Thomas <albertthoma...@gmail.com
<mailto:albertthoma...@gmail.com>> wrote:
When I was reading Sebastian's blog posts on Cross Validation a
few weeks ago I also found the example of Nested cross validation
on scikit-learn. At first like Daniel I thought the example was
not doing what it should be doing. But after a few minutes I
finally realized that it was correct. So I am for a bit more
clarification.
Albert
On Tue, 29 Nov 2016 at 02:53, Sebastian Raschka
<se.rasc...@gmail.com <mailto:se.rasc...@gmail.com>> wrote:
On first glance, the image shown in the image and the code
example seem to do/show the same thing? Maybe it would be
worth adding an explanatory figure like this to the docs to
clarify?
> On Nov 28, 2016, at 7:07 PM, Joel Nothman
<joel.noth...@gmail.com <mailto:joel.noth...@gmail.com>> wrote:
>
> If that clarifies, please offer changes to the example (as a
pull request) that make this clearer.
>
> On 29 November 2016 at 11:06, Joel Nothman
<joel.noth...@gmail.com <mailto:joel.noth...@gmail.com>> wrote:
> Briefly:
>
> clf = GridSearchCV(estimator=svr, param_grid=p_grid,
cv=inner_cv)
> nested_score = cross_val_score(clf, X=X_iris, y=y_iris,
cv=outer_cv)
>
> Each train/test split in cross_val_score holds out test
data. GridSearchCV then splits each train set into
(inner-)train and validation sets. There is no leakage of test
set knowledge from the outer loop into the grid search
optimisation; no leakage of validation set knowledge into the
SVR optimisation. The outer test data are reused as training
data, but within each split are only used to measure
generalisation error.
>
> Is that clear?
>
> On 29 November 2016 at 10:30, Daniel Homola
<dani.hom...@gmail.com <mailto:dani.hom...@gmail.com>> wrote:
> Dear all,
>
> I was wondering if the following example code is valid:
>
http://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html
<http://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html>
>
> My understanding is, that the point of nested
cross-validation is to prevent any data leakage from the inner
grid-search/param optimization CV loop into the outer model
evaluation CV loop. This could be achieved if the outer CV
loop's test data is completely separated from the inner loop's
CV, as shown here:
>
https://mlr-org.github.io/mlr-tutorial/release/html/img/nested_resampling.png
<https://mlr-org.github.io/mlr-tutorial/release/html/img/nested_resampling.png>
>
> The code in the above example however doesn't seem to
achieve this in any way.
>
> Am I missing something here?
>
> Thanks a lot,
> dh
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org <mailto:scikit-learn@python.org>
> https://mail.python.org/mailman/listinfo/scikit-learn
<https://mail.python.org/mailman/listinfo/scikit-learn>
>
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn@python.org <mailto:scikit-learn@python.org>
> https://mail.python.org/mailman/listinfo/scikit-learn
<https://mail.python.org/mailman/listinfo/scikit-learn>
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org <mailto:scikit-learn@python.org>
https://mail.python.org/mailman/listinfo/scikit-learn
<https://mail.python.org/mailman/listinfo/scikit-learn>
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org <mailto:scikit-learn@python.org>
https://mail.python.org/mailman/listinfo/scikit-learn
<https://mail.python.org/mailman/listinfo/scikit-learn>
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
