Dear Wiki user,
You have subscribed to a wiki page or wiki category on Pig Wiki for change
notification.
The following page has been changed by Shravan Narayanamurthy:
http://wiki.apache.org/pig/PigStagificationAndMRInfrastructure
New page:
= Stagification =
Stagification involves breaking the physical plan into multiple stages where
stage boundaries are drawn between the Local and Global Rearrange operators.
Stages are aggregated into MR jobs by taking two jobs at a time if there are
two or more than two stages. The remainder stage at the end will be executed
either as a map-only job or another map-reduce job. The so formed MR jobs will
be used to create a JobControl object with the relevant dependencies. Following
is an example:
{{{
A = load 'a';
B = foreach A generate $0, $2;
C = filter B by $2 10;
D = filter B by $2 10;
E = group C by $0, D by $0;
F = foreach E generate group, count(C), count(D);
}}}
