Cheolsoo Park created PIG-3959: ---------------------------------- Summary: Skewed join followed by replicated join fails in Tez Key: PIG-3959 URL: https://issues.apache.org/jira/browse/PIG-3959 Project: Pig Issue Type: Sub-task Components: tez Affects Versions: tez-branch Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: tez-branch
To reproduce the issue, run the following query- {code} x = LOAD 'foo' AS (x:int, y:chararray); y = LOAD 'bar' AS (x:int, y:chararray); a = JOIN x BY x, y BY x USING 'skewed'; z = LOAD 'zoo' AS (x:int, y:chararray); b = JOIN a BY x::x, z BY x USING 'replicated'; DUMP b; {code} This fails at runtime with the following error- {code} : Container released by application, AttemptID:attempt_1399657418038_0357_1_04_000000_3 Info:Error: org.apache.pig.backend.executionengine.ExecException: ERROR 2135: Received error from POLocalRearrage function.wrong key class: class org.apache.pig.impl.io.NullableIntWritable is not class org.apache.pig.impl.io.NullablePartitionWritable : at org.apache.pig.backend.hadoop.executionengine.tez.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:175) : at org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.runPipeline(PigProcessor.java:276) : at org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.run(PigProcessor.java:175) : at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307) : at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:581) : at java.security.AccessController.doPrivileged(Native Method) : at javax.security.auth.Subject.doAs(Subject.java:415) : at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) : at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:570) : Caused by: java.io.IOException: wrong key class: class org.apache.pig.impl.io.NullableIntWritable is not class org.apache.pig.impl.io.NullablePartitionWritable : at org.apache.tez.runtime.library.common.sort.impl.IFile$Writer.append(IFile.java:212) : at org.apache.tez.runtime.library.broadcast.output.FileBasedKVWriter.write(FileBasedKVWriter.java:149) : at org.apache.pig.backend.hadoop.executionengine.tez.POLocalRearrangeTez.getNextTuple(POLocalRearrangeTez.java:160) : ... 8 more {code} -- This message was sent by Atlassian JIRA (v6.2#6252)