srowen commented on a change in pull request #26596: [SPARK-29959][ML][PYSPARK]
Summarizer support more metrics
URL: https://github.com/apache/spark/pull/26596#discussion_r351326163
##########
File path: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala
##########
@@ -460,21 +486,52 @@ private[ml] object SummaryBuilderImpl extends Logging {
val realMean = Array.ofDim[Double](n)
var i = 0
while (i < n) {
- realMean(i) = currMean(i) * (weightSum(i) / totalWeightSum)
+ realMean(i) = currMean(i) * (currWeightSum(i) / totalWeightSum)
i += 1
}
Vectors.dense(realMean)
}
+ /**
+ * Sum of each dimension.
+ */
+ def sum: Vector = {
+ require(requestedMetrics.contains(Sum))
+ require(totalWeightSum > 0, s"Nothing has been added to this
summarizer.")
Review comment:
Well, the answer is certainly a vector of 0s in this case. It's different
from central moments where there is no value when there is no data. I don't
feel strongly about it though.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
