I believe it should null them out.
2011/5/25 Arun Chandy Thomas <[email protected]>
> Hi ,
>
> I am trying to use pig to aggregate data from an applications log lines.
>
> Most of the data in the input file have the following format:
> A B C D E F
>
> I am aggregating the data as follows:
>
> A= load '$in_dir' using PigStorage('\t') as (A, B,C,D,E,F);
> D = group A by (A, B,C,D,E,F);
> E = FOREACH D GENERATE FLATTEN(group) as (A, B,C,D,E,F ),COUNT(A) as hit
> STORE E INTO '$in_dir._1' using PigStorage('\t');
>
> In some cases i see the input lines are only : A B C D
> (E,F columns are missing)
> Would the pig script ignore such lines.
>
> Thanks & Regards,
> Arun
>
