[
https://issues.apache.org/jira/browse/TEZ-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638815#comment-16638815
]
Rohini Palaniswamy commented on TEZ-3983:
-----------------------------------------
Would be good to fix this case. Currently we have been switching to a unordered
partitioned shuffle edge in Pig
([https://github.com/apache/pig/blob/5b0a0bfaefad0d57c6c9fbbc09d709d13ea9af93/src/org/apache/pig/backend/hadoop/executionengine/tez/plan/optimizer/UnionOptimizer.java#L635-L642)]
for this scenario.
> VertexGroup ONE_TO_ONE edges do not produce the right graph
> -----------------------------------------------------------
>
> Key: TEZ-3983
> URL: https://issues.apache.org/jira/browse/TEZ-3983
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Adrian Nicoara
> Priority: Major
> Attachments:
> 0001-TEZ-3893.-VertexGroup-ONE_TO_ONE-DAG-verify-patched-.patch
>
>
> A VertexGroup is described to act as a union of multiple vertices.
> Consider a VertexGroup composed of 2 vertices, each with 2 tasks. If this is
> a union, then the expectation is that the VertexGroup has 4 tasks. A
> ONE_TO_ONE edge, to a downstream vertex, must then require that vertex to
> also have 4 tasks.
> This is not the case, as evidenced by the [following
> test|https://github.com/apache/tez/blob/261bbdd5929d562758deb31085b565db8e92d6a2/tez-api/src/test/java/org/apache/tez/dag/api/TestDAGVerify.java#L779-L831].
> Rather, what happens is that implementation (of unrolling the VertexGroup and
> directly creating edges between the components of the VertexGroup and the
> downstream vertices) is presented as the behavior.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
