[
https://issues.apache.org/jira/browse/TEZ-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147459#comment-14147459
]
Bikas Saha commented on TEZ-1612:
---------------------------------
Whats happening is that we recently change auto parallelism to wait for some
threshold of output data to be written (in addition to waiting for some
threshold of source tasks to complete) and that is delaying auto reduce
calculation till all source tasks complete. This is serializing 106 before 107
because 106 is the source of 107. Thus the race is gone.
> Pig on tez unit test intermittent hang
> --------------------------------------
>
> Key: TEZ-1612
> URL: https://issues.apache.org/jira/browse/TEZ-1612
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Daniel Dai
> Assignee: Bikas Saha
> Attachments: DAG1.png, runwithmaster.tar.gz,
> syslog_dag_1411413615885_0001_1, testfail1.log.tar.gz
>
>
> Several Pig unit tests hang intermittently. For example,
> TestNewPlanImplicitSplit.testImplicitSplitInCoGroup, which is a DAG of 4
> nodes:
> !DAG1.png!
> It uses auto-parallelism, vertex 106 change parallelism from 2->1, and vertex
> 107 from 21->1.
> Log attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
