Rajesh Balamohan created TEZ-2251:
-------------------------------------
Summary: Enabling auto reduce parallelism in certain jobs causes
DAG to hang
Key: TEZ-2251
URL: https://issues.apache.org/jira/browse/TEZ-2251
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Rajesh Balamohan
Scenario:
- Run TPCH query20
(https://github.com/cartershanklin/hive-testbench/blob/master/sample-queries-tpch/tpch_query20.sql)
at 1 TB scale (tez-master branch, hive trunk)
- Enable auto reduce parallelism
- DAG didn't complete and got stuck in "Reducer 6"
Vertex parallelism of "Reducer 5 & 6" happens within a span of 3 milliseconds,
and tasks of "reducer 5" ends up producing wrong partition details as it sees
the updated task numbers of reducer 6 when scheduled. This causes, job to hang.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
