[
https://issues.apache.org/jira/browse/SPARK-6004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337624#comment-14337624
]
Joseph K. Bradley commented on SPARK-6004:
------------------------------------------
We want to stop early because a priori we may have no idea if we should train
for 20 iterations or 1000 iterations. If we can stop early at 20 iterations,
that would be much better than wasting 50x more compute time. (That's extreme,
but you get the idea.) Keeping validationTol will still let users do what you
want to do (training for a lot of iterations and then pruning the model
afterwards).
> Pick the best model when training GradientBoostedTrees with validation
> ----------------------------------------------------------------------
>
> Key: SPARK-6004
> URL: https://issues.apache.org/jira/browse/SPARK-6004
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Liang-Chi Hsieh
> Priority: Minor
>
> Since the validation error does not change monotonically, in practice, it
> should be proper to pick the best model when training GradientBoostedTrees
> with validation instead of stopping it early.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
