Shubham Chopra updated PIG-637:

    Attachment: 637.patch

This happens because the optimizer eliminates the limit after a sort and puts 
an attribute in POSort/LOSort instead. This attribute is not used in the local 
mode sorting as this would adversely affect the MR sorting of the samples.

I have modified the code to avoid that optimization happening when executing in 
the local mode. I have also added a couple of test cases that verify the plans 
in both local and MR mode.

> limit with order by is broken in local mode
> -------------------------------------------
>                 Key: PIG-637
>                 URL: https://issues.apache.org/jira/browse/PIG-637
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>            Assignee: Shubham Chopra
>         Attachments: 637.patch
> Shubham, could you take a look.
> The following script when ran in local mode just ignores the limit and 
> outputs the entire data set:
> a = load 'studenttab10k' as (name, age,gpa);
> b = order a by name;
> c = limit b 10;
> dump c;
> The same script works fine in MR mode

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to