Hi Guys,
I am testing out AVRO in our cluster and having java.lang.NoSuchMethodError:
org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
when running a simple pig script.
after took a look at AVRO-493 I uploaded the new jackson jar to replace the
CDH2's jackson 1.0.1 jars.
The pigscipt looks like below
REGISTER /home/pig/jars/avro-1.4.1.jar
REGISTER /home/pig/jars/json_simple-1.1.jar
REGISTER /home/pig/jars/piggybank.jar
REGISTER /usr/lib/hadoop/lib/jackson-core-asl-1.5.5.jar
REGISTER /usr/lib/hadoop/lib/jackson-mapper-asl-1.5.5.jar
log_load = LOAD '/user/felix/avro_input/*.avro' USING
org.apache.pig.piggybank.storage.avro.AvroStorage() ;
dump log_load
jackson jars are copied to each slaves and master
The full stacktrace:
ERROR 2998: Unhandled internal error.
org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
java.lang.NoSuchMethodError:
org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
at org.apache.avro.Schema.<clinit>(Schema.java:82)
at
org.apache.pig.piggybank.storage.avro.ASCommons.<clinit>(ASCommons.java:44)
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:177)
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:133)
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:108)
at
org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:233)
at
org.apache.pig.impl.logicalLayer.LOLoad.determineSchema(LOLoad.java:169)
at
org.apache.pig.impl.logicalLayer.LOLoad.getSchema(LOLoad.java:150)
at
org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:843)
at
org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114)
at org.apache.pig.PigServer.registerQuery(PigServer.java:425)
at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
at org.apache.pig.Main.main(Main.java:357)
I am wondering if I need to restart tasktracker and jobtracker in order for
the jackson jars to be picked up?
