[
https://issues.apache.org/jira/browse/TEZ-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305783#comment-14305783
]
Bikas Saha commented on TEZ-2020:
---------------------------------
The exception is doing what its supposed to. All 1-1 connections must have
identical parallelism. Hence, auto-reduce changing the parallelism of 1 of the
1-1 edges to 1 is making it inconsistent with the other edge which is still at
10. Thus the destination of both 1-1 edges cannot proceed. So it throws an
exception. This is by design. The user must be careful about enabling
auto-reduce parallelism when such cardinality constraints exist downstream.
To summarize the exception thrown is correct and by design. In fact
TestInputReadyVertexManager.testComplex() checks for this.
> For 1-1 edge vertex configured event may be sent incorrectly
> ------------------------------------------------------------
>
> Key: TEZ-2020
> URL: https://issues.apache.org/jira/browse/TEZ-2020
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2020.1.patch, synthetic_job.png
>
>
> For a fully configured 1-1 vertex configured event will be sent after init
> because the vertex manager does not signal intent to reconfigure. However a
> change in parallelism upstream on the 1-1 edge can cause this vertex's
> parallelism to be updated accordingly. This breaks the guarantee that
> configured should be sent after vertex is fully configured. This is because
> update of parallelism on 1-1 edge has until now been hard-coded in the Vertex
> code. This can be removed now with the InputReadyVertexManager handling the
> 1-1 edge and the addition of vertex state updates. Not only will this fix the
> issue but also remove the special handling for 1-1 edges in the AM which
> should not be there for cleanliness.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
