Hi crunch !
When running MRPipeline here:
https://github.com/jayunit100/bigpetstore/blob/63958ddde9eb18ea75f078668a0023288547bbcf/src/integration/java/org/bigtop/bigpetstore/integration/BigPetStoreCrunchIT.java
I got two errors:
1) Running the code as-is, I got this very strange stacktrace:
Exception in thread "Thread-3" java.lang.VerifyError: class
org.apache.hadoop.yarn.proto.YarnProtos$URLProto overrides final method
getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
...
So, to debug... I tried something else and got another strange error.....
2) I changed line 45 to a "MemPipeline", and a new error took its place:
java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
at org.apache.crunch.DoFn.getConfiguration(DoFn.java:127)
at
org.apache.crunch.fn.Aggregators$AggregatorCombineFn.initialize(Aggregators.java:471)
at ...
I assume there is something wrong with my environment or job setup, but ive
logically used most of this code before, and am just refactoring, so am
puzzled why all these funny errors have cropped up.
any thoughts or leads would be very much appreciated ....
--
Jay Vyas
http://jayunit100.blogspot.com
