[
https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586832#action_12586832
]
Shravan Matthur Narayanamurthy commented on PIG-162:
----------------------------------------------------
Well anything that is non-blocking and occurs right after the cogroup will be
pushed into the reduce phase. For ex.,
A = load 'A' as (x,y);
B = group A by $0;
C = foreach B generate group,COUNT(A);
D = filter C by $1>10;
store D into 'D';
Here the foreach and filter will be pushed into the reduce phase so that there
is only one Map-Reduce job. The load and local-rearrange will occurr inside the
map and the package, foreach & filter will be in the reduce phase.
Hope this helps and gets me more help :)
> Rework mapreduce submission and monitoring
> ------------------------------------------
>
> Key: PIG-162
> URL: https://issues.apache.org/jira/browse/PIG-162
> Project: Pig
> Issue Type: Sub-task
> Environment: This bug tracks works to rework the submission and
> monitoring interface to map reduce as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: split.png
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
