[
https://issues.apache.org/jira/browse/PIG-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741764#comment-13741764
]
Cheolsoo Park commented on PIG-3410:
------------------------------------
Let me run e2e test.
I played with your patch a bit, and it works well for me. Basically, I tested
the following cases:
{code:title=LIMIT is not pushed down}
a = LOAD 'prodhive.cheolsoop.foo' USING MyStorage();
b = FILTER a BY part == 1 and nonpart == 'a';
c = LIMIT b 1;
DUMP c;
{code}
{code:title=LIMIT is pushed down}
a = LOAD 'prodhive.cheolsoop.foo' USING MyStorage();
b = FILTER a BY part == 1;
c = LIMIT b 1;
DUMP c;
{code}
> LimitOptimizer is applied before PartitionFilterOptimizer
> ---------------------------------------------------------
>
> Key: PIG-3410
> URL: https://issues.apache.org/jira/browse/PIG-3410
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11.1
> Reporter: Aniket Mokashi
> Assignee: Aniket Mokashi
> Attachments: PIG-3410.patch
>
>
> Consider following script-
> {code}
> hcat_load = LOAD 'X' using org.apache.hcatalog.pig.HCatLoader();
> hcat_filter = FILTER hcat_load BY (part='Y');
> hcat_limited = limit hcat_filter 5;
> dump hcat_limited;
> {code}
> This script is not benefited from LimitOptimizer (pushing limit to loadfunc)
> because LimitOptimizer is applied before PartitionFilterOptimizer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
