[jira] [Commented] (PIG-3743) Use VertexGroup and Alias vertex for union

[Rohini Palaniswamy (JIRA)]

    [ 
https://issues.apache.org/jira/browse/PIG-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948023#comment-13948023
 ] 

Rohini Palaniswamy commented on PIG-3743:
-----------------------------------------

[~cheolsoo],
 
POLocalRearrangeTez.java    
{code}
if (isUnion) {
                        // Use the entire tuple as both key and value
                        key = 
HDataType.getWritableComparableTypes(result.get(1), keyType);
                        val = new NullableTuple((Tuple)result.get(1));
                    }
{code}

  This seems very inefficient as map output size just doubles and the key is 
just discarded. Can we use POValueOutputTez instead of POLocalRearrangeTez and 
use RoundRobinPartitioner instead.

> Use VertexGroup and Alias vertex for union
> ------------------------------------------
>
>                 Key: PIG-3743
>                 URL: https://issues.apache.org/jira/browse/PIG-3743
>             Project: Pig
>          Issue Type: Sub-task
>          Components: tez
>            Reporter: Rohini Palaniswamy
>            Assignee: Cheolsoo Park
>             Fix For: tez-branch
>
>         Attachments: PIG-3743-1.patch, PIG-3743-2.patch, 
> PIG-3743-fix-skew.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)