[
https://issues.apache.org/jira/browse/PIG-362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olga Natkovich updated PIG-362:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
patch committed. thanks daniel for contributing!
> Limit can not push in front of ForEach with flatten
> ---------------------------------------------------
>
> Key: PIG-362
> URL: https://issues.apache.org/jira/browse/PIG-362
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: types_branch
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: types_branch
>
> Attachments: PIG-362-2.patch, PIG-362.patch
>
>
> Currently logical optimizer will push Limit in front of ForEach with flatten.
> It is based on the assumption that ForEach with a flatten always increase the
> number of records. However, this is a false assumption. In the case that
> there is empty bags inside input tuple, the number of output records can be
> 0, thus less than input records.
> We have no way to know whether there is an empty bag in the input at
> optimization time. So the only solution is not to push Limit in front of
> ForEach with flatten
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
