srowen commented on a change in pull request #26679: [SPARK-30044][ML]
MNB/CNB/BNB use empty sigma matrix instead of null
URL: https://github.com/apache/spark/pull/26679#discussion_r351563293
##########
File path:
mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala
##########
@@ -590,21 +588,17 @@ object NaiveBayesModel extends
MLReadable[NaiveBayesModel] {
instance.getModelType match {
case Multinomial | Bernoulli | Complement =>
- // Save model data: pi, theta
- require(instance.sigma == null)
- val data = Data(instance.pi, instance.theta)
-
sparkSession.createDataFrame(Seq(data)).repartition(1).write.parquet(dataPath)
-
+ require(instance.sigma.numRows == 0 && instance.sigma.numCols == 0)
Review comment:
Would you get an NPE if loading a model saved with 2.4, where it's null? or
am I missing that 2.4 would not have had this behavior (i.e. this is all new)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
