zhengruifeng commented on pull request #30548:
URL: https://github.com/apache/spark/pull/30548#issuecomment-737632676
slicing:
```
test("slicing vs non-slicing") {
val n = 10000
val size = 100
val floatArray = Array.tabulate(n * size)(i => i.toFloat)
val tic0 = System.nanoTime()
Seq.range(0, 1000).foreach { i =>
Seq.range(0, n).foreach { j =>
floatArray.slice(j * size, j * size + size).map(_.toDouble)
}
}
val toc0 = System.nanoTime()
val tic1 = System.nanoTime()
Seq.range(0, 1000).foreach { i =>
Seq.range(0, n).foreach { j =>
val doubles = Array.ofDim[Double](size)
val offset = j * size
var k = 0
while (k < size) { doubles(k) = floatArray(offset + k); k += 1 }
}
}
val toc1 = System.nanoTime()
println(s"slicing: ${toc0 - tic0}, non-slicing: ${toc1 - tic1}")
}
```
@srowen slicing and then mapping to double, is about 10X slower than the
new impl. It is somewhat surprising to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
