[
https://issues.apache.org/jira/browse/SPARK-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204762#comment-14204762
]
Sean Owen commented on SPARK-1227:
----------------------------------
OK you're interested in detecting overfitting, for one? You can't compare
algorithms with a different objective function this way though, right? a
different regularization param alone means a different objective. So I don't
see it can be a general benchmark, and other classifier metrics are more
appropriate. But yeah you could compare two runs that share the exact same
objective, and that's a way of comparing training algorithms.
Accuracy is in MulticlassMetrics which can be used with binary classification
but it would be a nice to have for BinaryClassificationMetrics.
> Diagnostics for Classification&Regression
> -----------------------------------------
>
> Key: SPARK-1227
> URL: https://issues.apache.org/jira/browse/SPARK-1227
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Martin Jaggi
> Assignee: Martin Jaggi
>
> Currently, the attained objective function is not computed (for efficiency
> reasons, as one evaluation requires one full pass through the data).
> For diagnostics and comparing different algorithms, we should however provide
> this as a separate function (one MR).
> Doing this requires the loss and regularizer functions themselves, not only
> their gradients (which are currently in the Gradient class). How about adding
> the new function directly on the corresponding models in classification/* and
> regression/* ? Any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
