[
https://issues.apache.org/jira/browse/TEZ-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635661#comment-14635661
]
Rohini Palaniswamy edited comment on TEZ-1190 at 7/21/15 7:21 PM:
------------------------------------------------------------------
[~bikassaha],
The classes are UnionOptimizer and TezDAGBuilder before PIG-4495 went in.
The code is not straightforward tez code though.
For eg:
A = LOAD 'data';
SPLIT A into B if $0 > 5, C if $0 > 10 and $0 < 12; // This is just for an
example. This simple condition can be written with just FILTER instead of split
and union.
D = UNION B, C;
E = GROUP D by $1;
The pig plan would be like: V1(Load) - > VG1, V1(Load) - >VG1, VG1 - >V2 (Group
by)
Vertex group VG1 takes two inputs from same source vertex V1. The output
vertex is only one i.e V2.
was (Author: rohini):
[~bikassaha],
The classes are UnionOptimizer and TezDAGBuilder before PIG-4495 went in.
The code is not straightforward tez code though.
For eg:
A = LOAD 'data';
SPLIT A into B if $0 > 5, C if $0 > 10 and $0 < 12; // This is just for an
example. This simple condition can be written with just FILTER instead of split
and union.
D = UNION B, C;
E = GROUP D by $1;
The pig plan would be like - V1(Load) -> VG1, V1(Load)->VG1, VG1->V2 (Group by)
Vertex group VG1 takes two inputs from same source vertex V1. The output
vertex is only one i.e V2.
> Allow multiple edges between two vertexes
> -----------------------------------------
>
> Key: TEZ-1190
> URL: https://issues.apache.org/jira/browse/TEZ-1190
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Daniel Dai
>
> This will be helpful in some scenario. In particular example, we can merge
> two small pipelines together in one pair of vertex. Note it is possible the
> edge type between the two vertexes are different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
