This is a bug. Looking at the code EqualTo isn't implemented for
Tuple, even though it is defined in the functional spec ( http://wiki.apache.org/pig/PigTypesFunctionalSpec
) and referenced in the user manual. Please file a JIRA on this so
we can track it and get it fixed. In the meantime, if you know your
tuple size, you can work around it by saying:
B = filter A by t1.a == t2.a and t1.b == t2.b;
Alan.
On Feb 3, 2010, at 1:23 AM, Neil Blue wrote:
Hello,
From the documentation I understand that it should be possible to
filter a
relation based on the equality of tuples.
http://hadoop.apache.org/pig/docs/r0.5.0/
piglatin_reference.html#deref:
...
Note 1: boolean (Tuple A is equal to tuple B if they have the same
size s,
and for all 0 <= i < s A[i] = = B[i])
...
However with this data file
-- indext.txt:
(1,one) (1,ONE)
(2,two) (22, twentytwo)
(3,three) (3,three)
I run this pig script:
A = LOAD 'indext.txt' AS (t1:(a:int, b:chararray), t2:(a:int,
b:chararray));
B = FILTER A BY t1==t2;
DUMP B;
Expecting the output:
((3,three),(3,three))
However there is an error:
2010-02-03 09:05:20,523 [main] ERROR
org.apache.pig.tools.grunt.Grunt -
ERROR 2067: EqualToExpr does not know how to handle type: tuple
Pig Stack Trace
---------------
ERROR 2067: EqualToExpr does not know how to handle type: tuple
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066:
Unable to
open iterator for alias B
at org.apache.pig.PigServer.openIterator(PigServer.java:475)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:
532)
at
org
.apache
.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.
java:190)
at
org
.apache
.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166
)
at
org
.apache
.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:142
)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:397)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR
1002:
Unable to store alias B
at org.apache.pig.PigServer.store(PigServer.java:530)
at org.apache.pig.PigServer.openIterator(PigServer.java:458)
... 6 more
Caused by: org.apache.pig.backend.executionengine.ExecException:
ERROR 2067:
EqualToExpr does not know how to handle type: tuple
at
org
.apache
.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.EqualToExpr.getNext(EqualToExpr.java:108)
at
org
.apache
.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POFilter.getNext(POFilter.java:148)
at
org
.apache
.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:231)
at
org
.apache
.pig.backend.local.executionengine.physicalLayer.counters.POCounte
r.getNext(POCounter.java:71)
at
org
.apache
.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator
.processInput(PhysicalOperator.java:231)
at
org
.apache
.pig.backend.hadoop.executionengine.physicalLayer.relationalOperat
ors.POStore.getNext(POStore.java:117)
at
org
.apache
.pig.backend.local.executionengine.LocalPigLauncher.runPipeline(Lo
calPigLauncher.java:146)
at
org
.apache
.pig.backend.local.executionengine.LocalPigLauncher.launchPig(Loca
lPigLauncher.java:109)
at
org
.apache
.pig.backend.local.executionengine.LocalExecutionEngine.execute(Lo
calExecutionEngine.java:165)
....
I am using pig 0.5 and hadoop .20.1.
Have I misunderstood the feature or is this a bug?
Thanks
Neil
*********************************************
The information contained in this message is likely to be
confidential. It is intended only for the person named above. Any
dissemination, distribution, copying, disclosure or use of this
message or its contents unless authorised by BioWisdom Ltd is
strictly prohibited. Any views or opinions expressed within this e-
mail are those of the author and do not necessarily represent those
of BioWisdom Ltd. If you have received this message in error, please
immediately notify us and delete it. Thank you. Registered Office:
BioWisdom Ltd, Harston Mill, Harston, Cambridge, CB22 7GG.
Registered in England: (GB) 3861669. VAT registered: (GB)
750899881. Tel: +44 (0)1223 874800, Fax: +44 (0) 1223 874801,
Internet:www.biowisdom.com
*********************************************
