Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change 
notification.

The following page has been changed by Shravan Narayanamurthy:
http://wiki.apache.org/pig/PigStagificationAndMRInfrastructure

New page:
= Stagification =
Stagification involves breaking the physical plan into multiple stages where 
stage boundaries are drawn between the Local and Global Rearrange operators. 
Stages are aggregated into MR jobs by taking two jobs at a time if there are 
two or more than two stages. The remainder stage at the end will be executed 
either as a map-only job or another map-reduce job. The so formed MR jobs will 
be used to create a JobControl object with the relevant dependencies. Following 
is an example:

{{{
A = load 'a';
B = foreach A generate $0, $2;
C = filter B by $2 < 10;
D = filter B by $2 > 10;
E = group C by $0, D by $0;
F = foreach E generate group, count(C), count(D);
}}}

Reply via email to