srowen commented on a change in pull request #26596: [SPARK-29959][ML][PYSPARK]
Summarizer support more metrics
URL: https://github.com/apache/spark/pull/26596#discussion_r350214561
##########
File path: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala
##########
@@ -460,21 +486,52 @@ private[ml] object SummaryBuilderImpl extends Logging {
val realMean = Array.ofDim[Double](n)
var i = 0
while (i < n) {
- realMean(i) = currMean(i) * (weightSum(i) / totalWeightSum)
+ realMean(i) = currMean(i) * (currWeightSum(i) / totalWeightSum)
i += 1
}
Vectors.dense(realMean)
}
+ /**
+ * Sum of each dimension.
+ */
+ def sum: Vector = {
+ require(requestedMetrics.contains(Sum))
+ require(totalWeightSum > 0, s"Nothing has been added to this
summarizer.")
Review comment:
Hm, do we need to require this? It's valid to ask for the sum over nothing
(0) in a way that isn't valid for variance / stdev. It is checked below, but
see comments below.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
