Rohini Palaniswamy updated PIG-5033:
    Attachment: PIG-5033-2.patch

   Sorry. Had to take over the patch as it required little more work. Hope that 
is fine with you.

Changes done
    - Inside of the union block, was not checking if the output from 
predecessor was to a scalar or replicate join. It always assumed the input went 
to POShuffleValueInputTez.
   - Changed TezCompilerUtil.isNonPackageInput to return true only for 
POFRJoinTez.  We only care about scalars and replicate join and it was 
returning true for POShuffleValueInputTez (union) as well.

> MultiQueryOptimizerTez creates bad plan with union, split and FRJoin
> --------------------------------------------------------------------
>                 Key: PIG-5033
>                 URL: https://issues.apache.org/jira/browse/PIG-5033
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>    Affects Versions: 0.16.0
>            Reporter: Travis Woodruff
>         Attachments: PIG-5033-2.patch, PIG-5033.patch, input1, input2, input3
> This script produces incorrect results:
> {code}
> a = load 'file:///tmp/input1' as (x:int, y:int);
> b = load 'file:///tmp/input2' as (x:int, y:int);
> u = union a,b;
> c = load 'file:///tmp/input3' as (x:int, y:int);
> e = filter c by y > 3;
> f = filter c by y < 2;
> g = join u by x left, e by x using 'replicated';
> h = join g by u::x left, f by x using 'replicated';
> store h into 'file:///tmp/pigoutput';
> {code}
> Without the union, or with opt.multiquery=false, or with non-replicated 
> joins, it works as expected.

This message was sent by Atlassian JIRA

Reply via email to