[GitHub] spark pull request #21120: [SPARK-22448][ML] Added sum function to Summerize...

[dedunumax]

Github user dedunumax closed the pull request at:

https://github.com/apache/spark/pull/21120


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21120: [SPARK-22448][ML] Added sum function to Summerize...

[dedunumax]

Github user dedunumax commented on a diff in the pull request:

https://github.com/apache/spark/pull/21120#discussion_r199135313
  
--- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala ---
@@ -562,6 +573,23 @@ private[ml] object SummaryBuilderImpl extends Logging {
 
   Vectors.dense(currL1)
 }
+
+/**
+ * Sum of each dimension
+ */
+def sum: Vector = {
+  require(requestedMetrics.contains(Sum))
+  require(totalWeightSum > 0, s"Nothing has been added to this 
summarizer.")
+
+  val realSum = Array.ofDim[Double](n)
+  var i = 0
+  val len = currMean.length
+  while (i < len) {
+realSum(i) = currMean(i) * weightSum(i)
+i += 1
--- End diff --

I will change that.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21120: [SPARK-22448][ML] Added sum function to Summerize...

[mahmoudmahdi24]

Github user mahmoudmahdi24 commented on a diff in the pull request:

https://github.com/apache/spark/pull/21120#discussion_r199119223
  
--- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala ---
@@ -562,6 +573,23 @@ private[ml] object SummaryBuilderImpl extends Logging {
 
   Vectors.dense(currL1)
 }
+
+/**
+ * Sum of each dimension
+ */
+def sum: Vector = {
+  require(requestedMetrics.contains(Sum))
+  require(totalWeightSum > 0, s"Nothing has been added to this 
summarizer.")
+
+  val realSum = Array.ofDim[Double](n)
+  var i = 0
+  val len = currMean.length
+  while (i < len) {
+realSum(i) = currMean(i) * weightSum(i)
+i += 1
--- End diff --

Please avoid using mutable values, use foldLeft for example to solve this.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21120: [SPARK-22448][ML] Added sum function to Summerize...

[dedunumax]

GitHub user dedunumax opened a pull request:

https://github.com/apache/spark/pull/21120

[SPARK-22448][ML] Added sum function to Summerizer and MultivariateOn…

…lineSummarizer

## What changes were proposed in this pull request?

This is going to add sum function to Summerizer and 
MultivariateOnlineSummarizer.

## How was this patch tested?

Added unit test to make sure it works.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dedunumax/spark SPARK-22448

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/21120.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #21120


commit 8c34fc9cfed27a3b53ead302088ab6f59e3690d4
Author: Dedunu Dhananjaya 
Date:   2018-04-21T08:24:19Z

[SPARK-22448][ML] Added sum function to Summerizer and 
MultivariateOnlineSummarizer




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org