Teng Peng created SPARK-24024:
---------------------------------
Summary: Fix deviance calculations in GLM to handle corner cases
Key: SPARK-24024
URL: https://issues.apache.org/jira/browse/SPARK-24024
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 2.3.0
Reporter: Teng Peng
It is reported by Spark users that the deviance calculations does not handle a
corner case. Thus, the correct model summary cannot be obtained. The user has
confirmed the the issue is in
override def deviance(y: Double, mu: Double, weight: Double): Double = {
2.0 * weight * (y * math.log(y / mu) - (y - mu))
}
when y = 0.
The user also mentioned there are many other places he believe we should check
the same thing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
