[
https://issues.apache.org/jira/browse/SPARK-22838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297859#comment-16297859
]
Apache Spark commented on SPARK-22838:
--------------------------------------
User 'ConeyLiu' has created a pull request for this issue:
https://github.com/apache/spark/pull/20026
> Avoid unnecessary copying of data
> ---------------------------------
>
> Key: SPARK-22838
> URL: https://issues.apache.org/jira/browse/SPARK-22838
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.2.1
> Reporter: Xianyang Liu
>
> If we read data from FileChannel to HeapByteBuffer, there is a need to copy
> the data from the off-heap to the on-heap, you can see the follow code:
> ```java
> static int read(FileDescriptor var0, ByteBuffer var1, long var2,
> NativeDispatcher var4) throws IOException {
> if(var1.isReadOnly()) {
> throw new IllegalArgumentException("Read-only buffer");
> } else if(var1 instanceof DirectBuffer) {
> return readIntoNativeBuffer(var0, var1, var2, var4);
> } else {
> ByteBuffer var5 = Util.getTemporaryDirectBuffer(var1.remaining());
> int var7;
> try {
> int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
> var5.flip();
> if(var6 > 0) {
> var1.put(var5);
> }
> var7 = var6;
> } finally {
> Util.offerFirstTemporaryDirectBuffer(var5);
> }
> return var7;
> }
> }
> ```
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
