Casting Exception in Illustrate for POForEach
---------------------------------------------
Key: PIG-2351
URL: https://issues.apache.org/jira/browse/PIG-2351
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.0, 0.10
Reporter: Jeremy Karn
When running illustrate on this script:
{code}
S1 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray,
query:chararray);
S2 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray,
query:chararray);
C = cogroup S1 BY user_id INNER, S2 by user_id INNER;
D = foreach C
generate group,
flatten(S1.timestamp) as t1,
flatten(S2.timestamp) as t2;
STORE D INTO 'output';
{code}
I get this exception:
{code}
2011-11-03 20:49:13,577 [main] ERROR org.apache.pig.PigServer -
java.lang.ClassCastException: java.lang.String cannot be cast to
org.apache.pig.data.Tuple
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.illustratorMarkup2(POForEach.java:683)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.createTuple(POForEach.java:458)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:404)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POOptimizedForEach.getNext(POOptimizedForEach.java:124)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POJoinPackage.getNext(POJoinPackage.java:222)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:416)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:399)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:1)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at
org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:215)
...
{code}
The casting error itself is easy to fix (by checking for type) - but that
results in the tracked lineages missing tuples. The method
POForEach.illustratorMarkup2 seems to be based on the incorrect assumption that
Object[] is an array of tuples instead of an array of fields for the output
tuple and its not clear to me if this function needs to be modified or if
there's a bigger problem somewhere else in the code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
