Mridul Muralidharan
Fri, 28 Mar 2008 10:49:08 -0700
Hi,As part of translating my generated pig query to something presentable here, I missed out the quotes around the zero, the semi-colon and other syntax issues which were not really central to my problem.
On further analysis, what I found was that the actual problem seems to be that I cant specify two different tuple arity as part of the conditional.
What I mean is, this wont work : -- start cut --A = load './a' USING PigStorage('\t') AS (id, name, source_id, target_id, score, atomic_target_val);
B = A;C = FILTER B by ( ( score > '0' ) and ( ( ( name eq 'Product_id' and atomic_target_val eq '1' and score > '0.5' ) ) or ( ( name eq 'type' and atomic_target_val eq 'Product' ) ) or ( ( name eq 'name' ) ) ) );
D = GROUP C by source_id;
E = FOREACH D {
F1 = FILTER C by ( name eq 'Product_id' and atomic_target_val eq
'1' and score > '0.5' ) ;
F2 = FILTER C by ( name eq 'type' and atomic_target_val eq
'Product' ) ;
GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0'
? '1' : '0' ) == '2' ? $1 : $0, $1) ; };
-- end cut -- or variants of it. While this will :GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0' ? '1' : '0' ) == '2' ? '-1' : $0), $1;
The arity of the generated tuple is always 2 - $1 is pulled out of the bincond - maybe this is not (supposed to be) supported ...
This is not really what I am looking for and so need another step to retrieve what I need ... but it is something to move forward with !
Thanks, Mridul Utkarsh Srivastava wrote:
I think you are missing a ";" before "}"C = FOREACH B { D1 = FILTER A by ( $0 eq 'val1' ) ; D2 = FILTER A by ( $1 eq 'val2' ) ; D3 = FILTER A by ( $2 eq 'val3' ) ; GENERATE ( ( ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?'1': '0' ) + ( COUNT (D3) > 0 ? '1' : '0' ) ) == '3' ? $1 : $0 )<HERE!!> };DUMP C;