Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19222
@rxin @cloud-fan I am very sorry for preparing performance data since I was
busy these weeks.
I confirmed that `MemoryBlock` approach is faster for OnHeap and is equal
to for OffHeap compared to `Platform` approach.
I ran [this benchmark
program](https://gist.github.com/kiszk/3d0db455166a7e2835ffdaf15c633f29) that
includes potentially polymorphic call for `MemoryBlock` like
```
def getMB(g: Int): MemoryBlock = {
if (g < mIters) {
byteArrayMB
} else if (g < mIters * 2) {
onHeapMB
} else if (g < mIters * 3) {
offHeapMB
...
} else {
null
}
}
benchmark.addCase("ByteArrayMemoryBlock getInt()") { t: Int =>
val mb = getMB(g)
if (t >= 0) g += 1
var sum = 0
for (_ <- 0L until iters) {
var i = 0
while (i < N / 4) {
sum += mb.getInt(i * 4)
i += 1
}
}
}
```
```
OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 on Linux
4.4.0-66-generic
Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz
Memory access benchmarks: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Platform getInt(Object byte[]) 1348 / 1355 199.1
5.0 1.0X
ByteArrayMemoryBlock getInt() 375 / 375 716.6
1.4 3.6X
OnHeapMemoryBlock getInt() 374 / 376 717.1
1.4 3.6X
Platform getInt(Object offHeap) 327 / 328 820.0
1.2 4.1X
OffHeapMemoryBlock get() 325 / 403 827.1
1.2 4.2X
Platform putInt(Object byte[]) 848 / 852 316.5
3.2 1.6X
ByteArrayMemoryBlock putInt() 571 / 571 470.4
2.1 2.4X
OnHeapMemoryBlock putInt() 575 / 575 467.2
2.1 2.3X
Platform putInt(Object offHeap) 582 / 584 461.0
2.2 2.3X
OffHeapMemoryBlock putInt() 583 / 584 460.1
2.2 2.3X
```
WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
