A cluster I'm using was recently upgraded to PIG 0.6. Since then,
I've been having problems with scripts that use PiggyBank functions.
All the map jobs for the script fail with:
WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.ClassCastException: org.apache.pig.ExecType cannot be cast
to org.apache.pig.impl.PigContext
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper.readFields(SliceWrapper.java:168)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:333)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)
INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
I compiled the PiggyBank jar using the latest code from SVN (as of Jan
9) and Pig 0.6. Below I've included a simple example program that
caused the error that simply reads a text file of words and lower
cases them.
register ./piggybank.jar
DEFINE ToLower org.apache.pig.piggybank.evaluation.string.LOWER();
words = LOAD './data/headwords_sample' USING PigStorage() as (word:charArray);
lowerCaseWords = FOREACH words GENERATE ToLower(word) as word;
STORE lowerCaseWords into './tmp/cooc3' USING PigStorage();
The Hadoop error isn't very informative about what is going on. Am I
using a compatible version of PiggyBank? What should I be doing
differently?
Thanks,
- Jeff
