[
https://issues.apache.org/jira/browse/TEZ-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-2834:
----------------------------------
Description:
Will attach the DAG.
Repro for reference: TPC-DS q_70 @ 30 TB scale.
"Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched
slightly late. But before "Reducer 9" can get scheduled, slots are taken up by
"Map 1", which is not preempted for running "Reducer 9".
This is with 0.7.1 codebase.
was:
Will attach the DAG.
Repro for reference: TPC-DS q_70 @ 30 TB scale.
"Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched
slightly late. But before "Reducer 9" can get scheduled, slots are taken up by
"Map 1", which is not preempted for running "Reducer 9".
> tez app hangs at large scale (~30TB)
> ------------------------------------
>
> Key: TEZ-2834
> URL: https://issues.apache.org/jira/browse/TEZ-2834
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Rajesh Balamohan
> Attachments: DAG_view.png, application_1442254312093_0095.1.log.gz,
> application_1442254312093_0095.2.log.gz, hive_view.png
>
>
> Will attach the DAG.
> Repro for reference: TPC-DS q_70 @ 30 TB scale.
> "Map 7" completes in 2 waves. Output is very tiny, so reducer 8 gets launched
> slightly late. But before "Reducer 9" can get scheduled, slots are taken up
> by "Map 1", which is not preempted for running "Reducer 9".
> This is with 0.7.1 codebase.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
