[
https://issues.apache.org/jira/browse/PIG-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836074#comment-13836074
]
Rohini Palaniswamy commented on PIG-3585:
-----------------------------------------
+1. The code is good if we have union after some processing. With this patch a
simple union takes 3 vertices (2 load and 1 union) instead of 1 vertex. We
should make it optimal to handle in a single vertex.
bq. Tez doesn't allow multiple inputs for the root vertex. We probably can't
implement UNION with a single vertex.
With MR, we just load both inputs as two different PigSplit in
PigInputFormat. That should be possible here too right?
> Make union work with tez
> ------------------------
>
> Key: PIG-3585
> URL: https://issues.apache.org/jira/browse/PIG-3585
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3585-1.patch
>
>
> Unlike MR, union needs to be implemented as multiple vertices because Tez
> doesn't allow multiple inputs in root vertex.
> I propose that we implement union by connecting load vertices to union vertex
> with broadcasting edges.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
