No, but you can make it by adding:
B = filter A by E is not null;
Alan.
On May 25, 2011, at 3:22 PM, Arun Chandy Thomas wrote:
Hi ,
I am trying to use pig to aggregate data from an applications log
lines.
Most of the data in the input file have the following format:
A B C D E F
I am aggregating the data as follows:
A= load '$in_dir' using PigStorage('\t') as (A, B,C,D,E,F);
D = group A by (A, B,C,D,E,F);
E = FOREACH D GENERATE FLATTEN(group) as (A, B,C,D,E,F ),COUNT(A) as
hit
STORE E INTO '$in_dir._1' using PigStorage('\t');
In some cases i see the input lines are only : A B C D (E,F columns
are missing)
Would the pig script ignore such lines.
Thanks & Regards,
Arun
