Thanks Dan for letting us know. That makes perfect sense because guava was updated to 11 in Pig 0.9, and Pig 0.8 had an old version guava.
On Wed, Mar 13, 2013 at 10:05 AM, Danfeng Li <[email protected]> wrote: > Ok, I figured out the problem. I have a customized hadoop conf directory. > In hadoop-env.sh file, I defined an "extra java classpath" as > > export > HADOOP_CLASSPATH="/usr/lib/hadoop/lib/*:/opt/hadoopgpl/lib/*:/opt/hadoopgpl/lib/pig-0.8.0/*:$HADOOP_CLASSPATH", > > so it's point to the jar file for pig 0.8. once I remove it, everything > works fine. > > Thanks Cheolsoo and Dmitriy for your help. > > Dan > > -----Original Message----- > From: Danfeng Li [mailto:[email protected]] > Sent: Tuesday, March 12, 2013 11:26 PM > To: [email protected] > Subject: RE: pig 0.12.0 ERROR 2998: Unhandled internal error. > com.google.common.collect.ImmutableSet.of > > Hi, Cheolsoo, > > I tried this on my laptop's ubuntu, it works fine. Do you have any > suggestion where should I look into on the server to figure out the problem? > > Thanks. > Dan > > -----Original Message----- > From: Cheolsoo Park [mailto:[email protected]] > Sent: Tuesday, March 12, 2013 11:13 PM > To: [email protected] > Subject: Re: pig 0.12.0 ERROR 2998: Unhandled internal error. > com.google.common.collect.ImmutableSet.of > > Hi Dan, > > I don't think you've downloaded the correct guava jar file. Here is the > output of javap on pig-withouthadoop.jar and google-guava-r11.jar: > > 1) pig-withouthadoop.jar > > /usr/java/latest/bin/javap -classpath pig-withouthadoop.jar -c > com.google.common.collect.ImmutableSet | grep 'public static > com.google.common.collect.ImmutableSet of' > public static com.google.common.collect.ImmutableSet of(); public static > com.google.common.collect.ImmutableSet of(java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object); public static com.google.common.collect.ImmutableSet > of(java.lang.Object, java.lang.Object, java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object, java.lang.Object, java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object); > public static com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, > java.lang.Object, java.lang.Object[]); > > 2) google-guava-r11.jar > /usr/java/latest/bin/javap -classpath google-guava-r11.jar -c > com.google.common.collect.ImmutableSet | grep 'public static > com.google.common.collect.ImmutableSet of' > public static com.google.common.collect.ImmutableSet of(); public static > com.google.common.collect.ImmutableSet of(java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object); public static com.google.common.collect.ImmutableSet > of(java.lang.Object, java.lang.Object, java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object, java.lang.Object, java.lang.Object); public static > com.google.common.collect.ImmutableSet of(java.lang.Object, > java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object); > public static com.google.common.collect.ImmutableSet of(java.lang.Object[]); > > As can be seen, the 7th static method is different. > > Now looking at your error message, it is complaining about that 7th static > method. > > >> java.lang.NoSuchMethodError: com.google.common.collect. > ImmutableSet.of(Ljava/lang/Object;Ljava/lang/Object; > Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object > [Ljava/lang/Object;)Lcom/google/common/collect/ImmutableSet; > > In fact, guava-11.0.jar should be embedded in pig-withouthadoop.jar, so I > am not sure why you're getting this error. > > Thanks, > Cheolsoo > > > > > > > > On Tue, Mar 12, 2013 at 4:45 PM, Danfeng Li <[email protected]> > wrote: > > > Here's the output for the secretDebugCmd > > > > dli2@sig1:~/$ pig12 -secretDebugCmd -param input="s" -param output="t" > > -c b224G_1.pig Find hadoop at /usr/bin/hadoop dry run: > > HADOOP_CLASSPATH: > > /home/dli2/github/pig/bin/../conf:/usr/lib/jvm/java-1.6.0-openjdk-1.6. > > 0.0.x86_64/lib/tools.jar:/etc/hadoop/hmaster_cluster/conf:/home/dli2/g > > ithub/pig/bin/../build/ivy/lib/Pig/jython-standalone-2.5.2.jar:/home/d > > li2/github/pig/bin/../build/ivy/lib/Pig/jruby-complete-1.6.7.jar:/home > > /dli2/github/pig/bin/../pig-withouthadoop.jar:./google-guava-r11.jar > > HADOOP_OPTS: -Xmx1000m > > -Dpig.log.dir=/home/dli2/github/pig/bin/../logs > > -Dpig.log.file=pig.log -Dpig.home.dir=/home/dli2/github/pig/bin/.. > > /usr/bin/hadoop jar /home/dli2/github/pig/bin/../pig-withouthadoop.jar > > -param input=s -param output=t -c b224G_1.pig > > > > Note the ./google-guava-r11.jar is in the class path. > > > > But I still got the same error. > > > > Thanks. > > Dan > > > > -----Original Message----- > > From: Dmitriy Ryaboy [mailto:[email protected]] > > Sent: Tuesday, March 12, 2013 4:30 PM > > To: [email protected] > > Subject: Re: pig 0.12.0 ERROR 2998: Unhandled internal error. > > com.google.common.collect.ImmutableSet.of > > > > Try this: > > > > pig -secretDebugCmd > > > > this will show you what's actually on the classpath. > > > > > > On Tue, Mar 12, 2013 at 3:32 PM, Danfeng Li <[email protected]> > > wrote: > > > > > I download version r11 ( > > > https://alexoutilities.googlecode.com/svn-history/r358/repository/co > > > m/ > > > google/common/google-guava/r11/google-guava-r11.jar) and put it in > > > my run directory, but still got the same error message. My CLASSPATH > > > is > > > > > > > > > CLASSPATH=.:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/lib/dt.jar:..... > > > > > > So it should look in the current directory, right? > > > > > > Thanks > > > Dan > > > > > > -----Original Message----- > > > From: Dmitriy Ryaboy [mailto:[email protected]] > > > Sent: Tuesday, March 12, 2013 2:58 PM > > > To: [email protected] > > > Subject: Re: pig 0.12.0 ERROR 2998: Unhandled internal error. > > > com.google.common.collect.ImmutableSet.of > > > > > > 11.0 is currently required. > > > > > > > > > On Tue, Mar 12, 2013 at 2:54 PM, Danfeng Li <[email protected]> > > > wrote: > > > > > > > Thanks for the quick repsonse, which guava version I should use? > > > > > > > > -----Original Message----- > > > > From: Dmitriy Ryaboy [mailto:[email protected]] > > > > Sent: Tuesday, March 12, 2013 2:52 PM > > > > To: [email protected] > > > > Subject: Re: pig 0.12.0 ERROR 2998: Unhandled internal error. > > > > com.google.common.collect.ImmutableSet.of > > > > > > > > Sounds like you have a bad (older? newer?) version of guava on > > > > the classpath. > > > > > > > > > > > > On Tue, Mar 12, 2013 at 2:50 PM, Danfeng Li > > > > <[email protected]> > > > > wrote: > > > > > > > > > When I try to run pig 0.12.0, I got the following error > > > > > > > > > > $ pig12 -param input="t" -param output="s" -c b224G_1.pig > > > > > log4j:ERROR Could not find value for key > > > > > log4j.appender.NullAppender log4j:ERROR Could not instantiate > > appender named "NullAppender". > > > > > 2013-03-12 17:44:17,724 [main] INFO org.apache.pig.Main - > > > > > Apache Pig version 0.12.0-SNAPSHOT (rexported) compiled Mar 07 > > > > > 2013, > > > > > 13:32:32 > > > > > 2013-03-12 17:44:17,725 [main] INFO org.apache.pig.Main - > > > > > Logging error messages to: /home/aaa/pig_1363124657723.log > > > > > 2013-03-12 17:44:17,934 [main] INFO > > > > > org.apache.pig.impl.util.Utils > > > > > - Default bootup file /home/aaa/.pigbootup not found > > > > > 2013-03-12 17:44:18,068 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > > > > > Connecting to hadoop file system at: hdfs://hmaster:8020 > > > > > 2013-03-12 17:44:18,215 [main] INFO > > > > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > > > > > Connecting to map-reduce job tracker at: hmaster:8021 > > > > > 2013-03-12 17:44:18,424 [main] ERROR > > > > > org.apache.pig.tools.grunt.Grunt > > > > > - ERROR 2998: Unhandled internal error. > > > > > com.google.common.collect.ImmutableSet.of(Ljava/lang/Object;Ljav > > > > > a/ > > > > > la > > > > > ng > > > > > /Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Lj > > > > > av > > > > > a/ > > > > > la > > > > > ng/Object;[Ljava/lang/Object;)Lcom/google/common/collect/Immutab > > > > > le Se t; Details at logfile: /home/aaa/pig_1363124657723.log > > > > > > > > > > In the log file > > > > > Pig Stack Trace > > > > > --------------- > > > > > ERROR 2998: Unhandled internal error. > > > > > com.google.common.collect.ImmutableSet.of(Ljava/lang/Object;Ljav > > > > > a/ > > > > > la > > > > > ng > > > > > /Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Lj > > > > > av > > > > > a/ > > > > > la > > > > > ng/Object;[Ljava/lang/Object;)Lcom/google/common/collect/Immutab > > > > > le > > > > > Se > > > > > t; > > > > > > > > > > java.lang.NoSuchMethodError: > > > > > > > > > com.google.common.collect.ImmutableSet.of(Ljava/lang/Object;Ljava/ > > > > la > > > > ng > > > > /Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljav > > > > a/ > > > > la > > > > ng/Object;[Ljava/lang/Object;)Lcom/google/common/collect/Immutable > > > > Se > > > > t; > > > > > at > > > > org.apache.pig.parser.QueryParser.<clinit>(QueryParser.java:287) > > > > > at > > > > > > > > > org.apache.pig.parser.QueryParserUtils.createParser(QueryParserUti > > > > ls > > > > .j > > > > ava:227) > > > > > at > > > > > > > > > org.apache.pig.parser.QueryParserUtils.createParser(QueryParserUti > > > > ls > > > > .j > > > > ava:223) > > > > > at > > > > > > > > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java > > > :2 > > > 25) > > > > > at > > > > > > > > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java > > > :1 > > > 74) > > > > > at > > > > > org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1632) > > > > > at > > > > > org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1605) > > > > > at > org.apache.pig.PigServer.registerQuery(PigServer.java:563) > > > > > at > > > > > > > > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1 > > > 03 > > > 2) > > > > > at > > > > > > > > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScr > > > > ip > > > > tP > > > > arser.java:499) > > > > > at > > > > > > > org.apache.pig.tools.grunt.GruntParser.loadScript(GruntParser.java:551 > > ) > > > > > at > > > > > > > > > org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser. > > > > ja > > > > va > > > > :340) > > > > > at > > org.apache.pig.tools.grunt.Grunt.checkScript(Grunt.java:98) > > > > > at org.apache.pig.Main.run(Main.java:600) > > > > > at org.apache.pig.Main.main(Main.java:157) > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorIm > > > > pl > > > > .j > > > > ava:57) > > > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAc > > > > ce > > > > ss > > > > orImpl.java:43) > > > > > at java.lang.reflect.Method.invoke(Method.java:616) > > > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:197) > > > > > > > > > > How should I fix it? > > > > > > > > > > Thanks. > > > > > Dan > > > > > > > > > > > > > > > > > > > > > > > > >
