sorry for the spamming, but I forgot to include the TaskTracker's process information.
root 20368 1 0 17:31 pts/0 00:00:00 su mapred -s /usr/java/default/bin/java -- -Dproc_tasktracker -Xmx1000m -Dhadoop.log.dir=/usr/lib/hadoop-0.20/logs -Dhadoop.log.file=hadoop-hadoop-tasktracker-ip-10-212-86-214.log -Dhadoop.home.dir=/usr/lib/hadoop-0.20 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64:/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath /etc/hadoop-0.20/conf:/usr/java/default/lib/tools.jar:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/hadoop-core-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20/lib/commons-daemon-1.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop-0.20/lib/commons-net-1.4.1.jar:/usr/lib/hadoop-0.20/lib/core-3.1.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-fairscheduler-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo-0.4.6.jar:/usr/lib/hadoop-0.20/lib/hadoop-thriftfs-0.20.2+737.jar:/usr/lib/hadoop-0.20/lib/hsqldb-1.8.0.10.jar: /usr/lib/hadoop-0.20/lib/jackson-core-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jackson-mapper-asl-1.5.5.jar:/usr/lib/hadoop-0.20/lib/jasper-compiler-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jasper-runtime-5.5.12.jar:/usr/lib/hadoop-0.20/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20/lib/jetty-6.1.14.jar:/usr/lib/hadoop-0.20/lib/jetty-util-6.1.14.jar:/usr/lib/hadoop-0.20/lib/junit-4.5.jar:/usr/lib/hadoop-0.20/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20/lib/log4j-1.2.15.jar:/usr/lib/hadoop-0.20/lib/mockito-all-1.8.2.jar:/usr/lib/hadoop-0.20/lib/mysql-connector-java-5.0.8-bin.jar:/usr/lib/hadoop-0.20/lib/oro-2.0.8.jar:/usr/lib/hadoop-0.20/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hadoop-0.20/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop-0.20/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop-0.20/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20/lib/jsp-2.1/jsp-api-2.1.jar:/usr/lib/hadoop-0.20/lib/hadoop-lzo.0.4.4.jar org.apache.hadoop.mapred.TaskTracker as you can see the correct version of jackson library is in the classpath. On Mon, Jan 24, 2011 at 2:09 PM, felix gao <[email protected]> wrote: > Chase, > > I tried to ran it on my local box with a standalone version of hadoop > installed and I still got the same error. > /usr/local/pig-0.7.0/bin/pig avro.pig > 11/01/24 14:01:29 INFO pig.Main: Logging error messages to: > /Users/felix/Documents/pig/pig_1295906489023.log > 2011-01-24 14:01:29,328 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting > to hadoop file system at: file:/// > 2011-01-24 14:01:30,277 [main] INFO > org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with > processName=JobTracker, sessionId= > 2011-01-24 14:01:30,537 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 2998: Unhandled internal error. > org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory; > > Any ideas why? > > Thanks, > > Felix > > > > > On Mon, Jan 24, 2011 at 1:18 PM, Chase Bradford > <[email protected]>wrote: > >> Yes, you will need to restart them. The child tasks inherit the TT's >> classpath, which will list only the 1.0.1 jars until you restart the >> daemon. >> >> On Mon, Jan 24, 2011 at 1:11 PM, felix gao <[email protected]> wrote: >> > Hi Guys, >> > >> > I am testing out AVRO in our cluster and having >> java.lang.NoSuchMethodError: >> > >> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory; >> > when running a simple pig script. >> > after took a look at AVRO-493 I uploaded the new jackson jar to replace >> the >> > CDH2's jackson 1.0.1 jars. >> > >> > >> > The pigscipt looks like below >> > REGISTER /home/pig/jars/avro-1.4.1.jar >> > REGISTER /home/pig/jars/json_simple-1.1.jar >> > REGISTER /home/pig/jars/piggybank.jar >> > REGISTER /usr/lib/hadoop/lib/jackson-core-asl-1.5.5.jar >> > REGISTER /usr/lib/hadoop/lib/jackson-mapper-asl-1.5.5.jar >> > >> > log_load = LOAD '/user/felix/avro_input/*.avro' USING >> > org.apache.pig.piggybank.storage.avro.AvroStorage() ; >> > >> > dump log_load >> > >> > >> > jackson jars are copied to each slaves and master >> > >> > The full stacktrace: >> > ERROR 2998: Unhandled internal error. >> > >> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory; >> > >> > java.lang.NoSuchMethodError: >> > >> org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory; >> > at org.apache.avro.Schema.<clinit>(Schema.java:82) >> > at >> > >> org.apache.pig.piggybank.storage.avro.ASCommons.<clinit>(ASCommons.java:44) >> > at >> > >> org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:177) >> > at >> > >> org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:133) >> > at >> > >> org.apache.pig.piggybank.storage.avro.AvroStorage.getAvroSchema(AvroStorage.java:108) >> > at >> > >> org.apache.pig.piggybank.storage.avro.AvroStorage.getSchema(AvroStorage.java:233) >> > at >> > org.apache.pig.impl.logicalLayer.LOLoad.determineSchema(LOLoad.java:169) >> > at >> > org.apache.pig.impl.logicalLayer.LOLoad.getSchema(LOLoad.java:150) >> > at >> > >> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:843) >> > at >> > >> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63) >> > at >> org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164) >> > at >> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114) >> > at org.apache.pig.PigServer.registerQuery(PigServer.java:425) >> > at >> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737) >> > at >> > >> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324) >> > at >> > >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) >> > at >> > >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) >> > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) >> > at org.apache.pig.Main.main(Main.java:357) >> > >> > I am wondering if I need to restart tasktracker and jobtracker in order >> for >> > the jackson jars to be picked up? >> > >> > >> > >> > >> > >> > >> > >> > >> >> >> >> -- >> Chase Bradford >> >> >> “If in physics there's something you don't understand, you can always >> hide behind the uncharted depths of nature. But if your program >> doesn't work, there is no obstinate nature. If it doesn't work, you've >> messed up.” >> >> - Edsger Dijkstra >> > >
