[
https://issues.apache.org/jira/browse/TEZ-1491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009894#comment-15009894
]
Rajesh Balamohan commented on TEZ-1491:
---------------------------------------
In merger case, this would be beneficial if we have many number of RLE_MARKER
in which case the reader's position would not change and localProgress would
not need to update the progress often. In cases where all keys are distinct,
it would still encounter the penalty of sync block.
lgtm. +1
> Tez reducer-side merge's counter update is slow
> -----------------------------------------------
>
> Key: TEZ-1491
> URL: https://issues.apache.org/jira/browse/TEZ-1491
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.6.0, 0.7.0, 0.8.0-alpha
> Reporter: Gopal V
> Assignee: Gopal V
> Labels: Performance
> Attachments: TEZ-1491.1.patch, perf-top-counters.png
>
>
> TezMerger$MergeQueue::next() shows up in profiles due a synchronized block in
> a tight loop.
> Part of the slow operation was due to DataInputBuffer issues identified
> earlier in HADOOP-10694, but along with that approx 11% of my lock prefix
> calls were originating from the following line.
> {code}
> mergeProgress.set(totalBytesProcessed * progPerByte);
> {code}
> in two places within the core loop.
> !perf-top-counters.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
