Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r175548118
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java ---
@@ -80,7 +76,7 @@ public void zeroOut() {
public void set(int index, long value) {
assert index >= 0 : "index (" + index + ") should >= 0";
assert index < length : "index (" + index + ") should < length (" +
length + ")";
- Platform.putLong(baseObj, baseOffset + index * WIDTH, value);
+ memory.putLong(memory.getBaseOffset() + index * WIDTH, value);
--- End diff --
@cloud-fan sorry for my delay. I was busy with several stuffs.
I ran a benchmark program for `LongArray.zeroOut` with two version. One is
current implementation. The other is future implementation like
`memory.putLong(index * WIDTH, value)`. I got almost the same performance using
two versions.
WDYT?
```
OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 on Linux
4.4.0-66-generic
Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz
Platform MemoryAccess: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
zeroOutCurrent 1066 / 1071 1259.4
0.8 1.0X
zeroOutFuture 1061 / 1063 1265.6
0.8 1.0X
```
```
class MemoryBlockLoopAccessBenchmark extends SparkFunSuite {
test("benchmark") {
val N = 128 * 1024 * 1024
val iters = 2
val M = 5
val benchmark = new Benchmark("Platform MemoryAccess", 1L * M * N *
iters,
minNumIters = 20)
val array = new Array[Long](N)
val mb = OnHeapMemoryBlock.fromArray(array)
val la = new LongArray(mb)
benchmark.addCase("zeroOutCurrent") { _: Int =>
for (_ <- 0L until iters) {
var i = 0
while (i < M) {
la.zeroOut()
i += 1
}
}
}
benchmark.addCase("zeroOutFuture") { _: Int =>
for (_ <- 0L until iters) {
var i = 0
while (i < M) {
la.zeroOutFuture()
i += 1
}
}
}
benchmark.run()
}
}
public final class LongArray {
...
public void zeroOut() {
long baseOffset = memory.getBaseOffset();
for (long off = baseOffset; off < baseOffset + length * WIDTH; off +=
WIDTH) {
memory.putLong(off, 0);
}
}
public void zeroOutFuture() {
for (long off = 0; off < length * WIDTH; off += WIDTH) {
memory.putLong2(off, 0);
}
}
...
}
public final class OnHeapMemoryBlock extends MemoryBlock {
...
@Override
public final void putLong(long offset, long value) {
Platform.putLong(array, offset, value);
}
@Override
public final void putLong2(long ofs, long value) {
Platform.putLong(array, ofs + offset, value);
}
...
}
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
