Re: Filter equality with tuple

[Alan Gates]

This is a bug.  Looking at the code EqualTo isn't implemented for  
Tuple, even though it is defined in the functional spec ( http://wiki.apache.org/pig/PigTypesFunctionalSpec 
 ) and referenced in the user manual.  Please file a JIRA on this so  
we can track it and get it fixed.  In the meantime, if you know your  
tuple size, you can work around it by saying:

B = filter A by t1.a == t2.a and t1.b == t2.b;

Alan.

On Feb 3, 2010, at 1:23 AM, Neil Blue wrote:

Hello,

From the documentation I understand that it should be possible to  
filter a
relation based on the equality of tuples.

http://hadoop.apache.org/pig/docs/r0.5.0/ 
piglatin_reference.html#deref:
...
Note 1: boolean (Tuple A is equal to tuple B if they have the same  
size s,
and for all 0 <= i < s A[i] = = B[i])
...

However with this data file

-- indext.txt:
(1,one) (1,ONE)
(2,two) (22, twentytwo)
(3,three)       (3,three)

I run this pig script:

A = LOAD 'indext.txt' AS (t1:(a:int, b:chararray), t2:(a:int,  
b:chararray));
B = FILTER A BY t1==t2;
DUMP B;

Expecting the output:
((3,three),(3,three))

However there is an error:
2010-02-03 09:05:20,523 [main] ERROR  
org.apache.pig.tools.grunt.Grunt -
ERROR 2067: EqualToExpr does not know how to handle type: tuple

Pig Stack Trace
---------------
ERROR 2067: EqualToExpr does not know how to handle type: tuple

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:  
Unable to
open iterator for alias B
       at org.apache.pig.PigServer.openIterator(PigServer.java:475)
       at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java: 
532)
       at
org 
.apache 
.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.
java:190)
       at
org 
.apache 
.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166
)
       at
org 
.apache 
.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:142
)
       at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
       at org.apache.pig.Main.main(Main.java:397)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR  
1002:
Unable to store alias B
       at org.apache.pig.PigServer.store(PigServer.java:530)
       at org.apache.pig.PigServer.openIterator(PigServer.java:458)
       ... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException:  
ERROR 2067:
EqualToExpr does not know how to handle type: tuple
       at
org 
.apache 
.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.EqualToExpr.getNext(EqualToExpr.java:108)
       at
org 
.apache 
.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POFilter.getNext(POFilter.java:148)
       at
org 
.apache 
.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:231)
       at
org 
.apache 
.pig.backend.local.executionengine.physicalLayer.counters.POCounte
r.getNext(POCounter.java:71)
       at
org 
.apache 
.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:231)
       at
org 
.apache 
.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POStore.getNext(POStore.java:117)
       at
org 
.apache 
.pig.backend.local.executionengine.LocalPigLauncher.runPipeline(Lo
calPigLauncher.java:146)
       at
org 
.apache 
.pig.backend.local.executionengine.LocalPigLauncher.launchPig(Loca
lPigLauncher.java:109)
       at
org 
.apache 
.pig.backend.local.executionengine.LocalExecutionEngine.execute(Lo
calExecutionEngine.java:165)
....

I am using pig 0.5 and hadoop .20.1.

Have I misunderstood the feature or is this a bug?

Thanks
Neil

*********************************************

The information contained in this message is likely to be  
confidential.  It is intended only for the person named above.  Any  
dissemination, distribution, copying, disclosure or use of this  
message or its contents unless authorised by BioWisdom Ltd is  
strictly prohibited. Any views or opinions expressed within this e- 
mail are those of the author and do not necessarily represent those  
of BioWisdom Ltd. If you have received this message in error, please  
immediately notify us and delete it.  Thank you.  Registered Office:  
BioWisdom Ltd, Harston Mill, Harston, Cambridge, CB22 7GG.   
Registered in England: (GB) 3861669.  VAT registered: (GB)  
750899881.  Tel: +44 (0)1223 874800, Fax: +44 (0) 1223 874801,  
Internet:www.biowisdom.com

*********************************************