zhengruifeng edited a comment on pull request #30548:
URL: https://github.com/apache/spark/pull/30548#issuecomment-737620983
test performance:
```
test("add float to array") {
val floatArrays = Array.tabulate(10000, 100)((i, j) => i.toFloat / (j +
1))
val vectors = floatArrays.map(array =>
Vectors.dense(array.map(_.toDouble)))
val tic0 = System.nanoTime()
Seq.range(0, 1000).foreach { i =>
val sum = Array.ofDim[Double](100)
floatArrays.foreach { array =>
var j = 0
while (j < 100) { sum(j) += array(j); j += 1 }
}
}
val toc0 = System.nanoTime()
val tic1 = System.nanoTime()
Seq.range(0, 1000).foreach { i =>
val sum = Vectors.zeros(100)
vectors.foreach { vec =>
org.apache.spark.ml.linalg.BLAS.axpy(1.0, vec, sum)
}
}
val toc1 = System.nanoTime()
println(s"array sum: ${toc0 - tic0}, vector axpy: ${toc1 - tic1}")
}
```
result:
@srowen it seems that directly adding float values is nearly 2x faster than
using `axpy`, while halving the broadcast size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
