[
https://issues.apache.org/jira/browse/SPARK-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204832#comment-14204832
]
Martin Jaggi commented on SPARK-1227:
-------------------------------------
yes, this can sometimes help to detect overfitting, if combined with test
error. and tells us if the model is over-trained or under-trained (such as when
using early stopping).
and yes, comparison of methods is only possible for the same objective function.
it's not a replacement for test error, but a benchmark suite would benefit if
it could keep track of both test and train objectives.
> Diagnostics for Classification&Regression
> -----------------------------------------
>
> Key: SPARK-1227
> URL: https://issues.apache.org/jira/browse/SPARK-1227
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Martin Jaggi
> Assignee: Martin Jaggi
>
> Currently, the attained objective function is not computed (for efficiency
> reasons, as one evaluation requires one full pass through the data).
> For diagnostics and comparing different algorithms, we should however provide
> this as a separate function (one MR).
> Doing this requires the loss and regularizer functions themselves, not only
> their gradients (which are currently in the Gradient class). How about adding
> the new function directly on the corresponding models in classification/* and
> regression/* ? Any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
