[
https://issues.apache.org/jira/browse/PIG-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150645#comment-13150645
]
Gianmarco De Francisci Morales commented on PIG-2367:
-----------------------------------------------------
I am getting those errors on Apache Pig version 0.9.1.1110111530 (r1182127)
I assume that you use that class of elephantbird in production at twitter, so
the error must be related with the combination of jobs/optimizations that my
script produces.
Looking quickly at the source code, I see in JobControlCompiler.java@629
nwJob.setOutputValueClass(NullableTuple.class);
Is this correct?
I will try to use a subclass of SequenceFileStorage from elephantbird to see
whether the error still comes up.
I can also share my StoreFunc, the script and a sample of data to reproduce the
issue.
> StoreFunc should work with OutputFormat<ComparableWritable,Writable>
> --------------------------------------------------------------------
>
> Key: PIG-2367
> URL: https://issues.apache.org/jira/browse/PIG-2367
> Project: Pig
> Issue Type: Bug
> Reporter: Gianmarco De Francisci Morales
>
> Right now, if one wants to write a SequenceFile from a custom StoreFunc he
> needs to declare it to be a OutputFormat<NullableText, NullableTuple>
> Otherwise you get these exceptions:
> {code}
> ERROR org.apache.pig.tools.grunt.GruntParser -
> ERROR 2997: Unable to recreate exception from backed error:
> java.io.IOException: java.io.IOException: wrong key class:
> org.apache.hadoop.io.NullWritable is not class
> org.apache.pig.impl.io.NullableText
> ERROR org.apache.pig.tools.grunt.GruntParser -
> ERROR 2997: Unable to recreate exception from backed error:
> java.io.IOException: java.io.IOException: wrong value class:
> org.apache.hadoop.io.BytesWritable is not class
> org.apache.pig.impl.io.NullableTuple
> {code}
> And stack trace:
> {code}
> java.io.IOException: java.io.IOException: wrong key class:
> org.apache.hadoop.io.NullWritable is not class org.apache.pig.impl.io.
> NullableText
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:464)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:427)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:399)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:261)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:255)
> Caused by: java.io.IOException: wrong key class:
> org.apache.hadoop.io.NullWritable is not class
> org.apache.pig.impl.io.NullableText
> at
> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:985)
> at
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:74)
> at mypackage.pig.BinStorage.putNext(BinStorage.java:75)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:462)
> ... 11 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
