[
https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900142#comment-14900142
]
Rajesh Balamohan commented on TEZ-2834:
---------------------------------------
lgtm. +1
Very minor:
{noformat}
!highestPriRequest.getPriority().equals(highestWaitingRequestPriority)
{noformat} be replaced with {noformat}
highestPriRequest.getPriority().compareTo(highestWaitingRequestPriority) != 0
{noformat} to avoid null & class checks in preemptIfNeeded().
> tez app hangs at large scale (~30TB)
> ------------------------------------
>
> Key: TEZ-2834
> URL: https://issues.apache.org/jira/browse/TEZ-2834
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Rajesh Balamohan
> Assignee: Bikas Saha
> Attachments: DAG_view.png, TEZ-2834.1.patch, TEZ-2834.2.patch,
> application_1442254312093_0095.1.log.gz,
> application_1442254312093_0095.2.log.gz, hive_view.png
>
>
> Will attach the DAG.
> Repro for reference: TPC-DS q_70 @ 30 TB scale.
> "Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched
> slightly late. But before "Reducer 9" can get scheduled, slots are taken up
> by "Map 1", which is not preempted for running "Reducer 9".
> This is with 0.7.1 codebase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
