Xianyang Liu created SPARK-22838:
------------------------------------
Summary: Avoid unnecessary copying of data
Key: SPARK-22838
URL: https://issues.apache.org/jira/browse/SPARK-22838
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.2.1
Reporter: Xianyang Liu
If we read data from FileChannel to HeapByteBuffer, there is a need to copy the
data from the off-heap to the on-heap, you can see the follow code:
```java
static int read(FileDescriptor var0, ByteBuffer var1, long var2,
NativeDispatcher var4) throws IOException {
if(var1.isReadOnly()) {
throw new IllegalArgumentException("Read-only buffer");
} else if(var1 instanceof DirectBuffer) {
return readIntoNativeBuffer(var0, var1, var2, var4);
} else {
ByteBuffer var5 = Util.getTemporaryDirectBuffer(var1.remaining());
int var7;
try {
int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
var5.flip();
if(var6 > 0) {
var1.put(var5);
}
var7 = var6;
} finally {
Util.offerFirstTemporaryDirectBuffer(var5);
}
return var7;
}
}
```
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
