pig-user  

Nested expressions in FOREACH vs FILTER

Mridul Muralidharan
Thu, 27 Mar 2008 15:39:02 -0700

Hi,

  FOREACH supports nested expressions of form :
var1 = FOREACH var { <expr>'s; GENERATE <tuple> }

Similar functionality does not seem to be available with FILTER.
That is, slightly complex filter expressions - particularly when we need to process the Bag/tuples contained as tuples of the relation in question is not possible.

Mirroring FOREACH functionality, something like this would be great :

var1 = FILTER var {
  t1 = <expr>;
  t2 = <expr>;
  ...
  BY (conds);
}


Workaround for the immediate problem I am facing is to use FOREACH to generate something like $status, <tuple> and then FILTER on $status.
Followed by another FOREACH to remove the status.

Regards,
Mridul