Re: GLM Poisson Model - Deviance calculations

[svattig]

Yes i’m referring to that method deviance. It fails when ever y is 0. I think
R deviance calculation logic checks if y is 0 and assigns 1 to y for such
cases.

There are few deviances Like nulldeviance, residualdiviance and deviance
that Glm regression summary object has.
You might want to check those as well so the toString method doesn’t fail.

Thank you,
Srikar.V



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

GLM Poisson Model - Deviance calculations

[svattig]

In Spark 2.3, When Poisson Model(with labelCol having few counts as 0's) is
fit, the Deviance calculations are broken as result of log(0). I think this
is the same case as in spark 2.2. 
But the new toString method in Spark 2.3's
GeneralizedLinearRegressionTrainingSummary class is throwing error at line
1551 with NumberFormatException. Due to this exception, we are not able to
get the summary object from Model fit.

Can the toString method be fixed including Deviance calculations for example
taking log(1) when ever the count is 0 instead of having log(0) ?

Thanks,
Srikar.V



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org