Jonathan Eagles created TEZ-3709:
------------------------------------
Summary: TezMerger is slow for high number of segments
Key: TEZ-3709
URL: https://issues.apache.org/jira/browse/TEZ-3709
Project: Apache Tez
Issue Type: Bug
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
The below code is a bad performer at scale since it has to memcpy the whole
list of segments for each item in the batch instead of of just once per batch.
This is true for both computeBytesInMerges and getSegmentDescriptors.
{code}
for (int i = 0; i < batch; i++) {
ArrayList#remove(0)
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
