[
https://issues.apache.org/jira/browse/TEZ-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169901#comment-15169901
]
Jason Lowe commented on TEZ-3140:
---------------------------------
There are other cases in Hadoop code where they need to do something similar,
and if we want to avoid the custom ByteBufferDataInput class I think we could
reuse the DataInputByteBuffer and accomplish the same no-copy semantics like
this:
{code}
if (out instanceof DataOutputBuffer) {
DataOutputBuffer dub = (DataOutputBuffer) out;
DataInputByteBuffer dibb = new DataInputByteBuffer();
dibb.reset(bb);
dub.write(dibb, size);
{code}
ShuffleUtils and DagTypeConverters are already doing similar things with
DataInputByteBuffer.
> Reduce AM memory usage while serialization
> ------------------------------------------
>
> Key: TEZ-3140
> URL: https://issues.apache.org/jira/browse/TEZ-3140
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.7.1, 0.8.3
>
> Attachments: TEZ-3140-1.patch
>
>
> There is an unnecessary copy of userpayload byte array during
> serialization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
