So, I finally found the problem... I added the line:
export HADOOP_CLASSPATH=$CLASSPATH:$HADOOP_CLASSPATH
to hadoop-env.sh and that fixed the classpath problem. I'm now able to
use the hive command line interface.
Perhaps you can make a note of that in the docs in case someone else
runs into the same problem?
Thanks for helping me debug this,
Shirley
Shirley Cohen wrote:
Hi Prasad,
I'm running java version "1.6.0_11"
Java(TM) SE Runtime Environment (build 1.6.0_11-b03)
Java HotSpot(TM) Server VM (build 11.0-b16, mixed mode)
I don't have a $HADOOP_CLASSPATH environment variable nor is hadoop in
my path. The only hadoop environment variable that I have set is
$HADOOP_HOME, which seems to be required by the hive script.
The output that I get from running the hive script is:
-bash-3.2$ ./hive
/users/scohen/hive/build/dist/bin/../conf:/users/scohen/hive/build/dist/bin/../lib/TestSerDe.jar:/users/scohen/hive/build/dist/bin/../lib/antlr-runtime-3.0.1.jar:/users/scohen/hive/build/dist/bin/../lib/asm-3.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/users/scohen/hive/build/dist/bin/../lib/commons-collections-3.2.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-jexl-1.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-lang-2.4.jar:/users/scohen/hive/build/dist/bin/../lib/commons-logging-1.0.4.jar:/users/scohen/hive/build/dist/bin/../lib/commons-logging-api-1.0.4.jar:/users/scohen/hive/build/dist/bin/../lib/derby.jar:/users/scohen/hive/build/dist/bin/../lib/hive_anttasks.jar:/users/scohen/hive/build/dist/bin/../lib/hive_cli.jar:/users/scohen/hive/build/dist/bin/../lib/hive_common.jar:/users/scohen/hive/build/dist/bin/../lib/hive_exec.jar:/users/scohen/hive/build/dist/bin/../lib/hive_jdbc.jar:/users/scohen/hive/build/dist/bin/../lib/hive_metastore.jar:/users/scohen/hive/build/dist/bin/../lib/hive_serde.jar:/users/scohen/hive/build/dist/bin/../lib/hive_service.jar:/users/scohen/hive/build/dist/bin/../lib/jdo2-api-2.1.jar:/users/scohen/hive/build/dist/bin/../lib/jline-0.9.94.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-core-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-enhancer-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-rdbms-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/json.jar:/users/scohen/hive/build/dist/bin/../lib/junit-3.8.1.jar:/users/scohen/hive/build/dist/bin/../lib/libfb303.jar:/users/scohen/hive/build/dist/bin/../lib/libthrift.jar:/users/scohen/hive/build/dist/bin/../lib/log4j-1.2.15.jar:/users/scohen/hive/build/dist/bin/../lib/stringtemplate-3.1b1.jar:/users/scohen/hive/build/dist/bin/../lib/velocity-1.5.jar:
java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hive.conf.HiveConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 7 more
I have echoed the last line, but the script errors out before reaching
the last line, so nothing gets printed out except for what I have
copied above.
Attached, is the hive script that I'm using (just in case that helps).
Please let me know if you have any idea what's going on. I agree that
this is weird!
Thanks,
Shirley
Prasad Chakka wrote:
Hi Shirley,
I am not sure what is happening here. Could you give us the following?
Java version (it should have java6)
HADOOP_CLASSPATH env value (should be nothing)
`which hadoop` (remove hadoop from PATH if it is there)
Echo the last line of bin/hive
------------------------------------------------------------------------
*From: *Shirley Cohen <[email protected]>
*Reply-To: *<[email protected]>
*Date: *Thu, 29 Jan 2009 18:29:27 -0800
*To: *<[email protected]>
*Subject: *Re: classpath issue
Yes, it's there. Here is the output from the jar command:
-bash-3.2$ ./jar tvf
/users/scohen/hive/build/dist/bin/../lib/hive_common.jar
0 Thu Jan 29 09:55:10 MST 2009 META-INF/
102 Thu Jan 29 09:55:08 MST 2009 META-INF/MANIFEST.MF
0 Thu Jan 29 09:55:10 MST 2009 org/
0 Thu Jan 29 09:55:10 MST 2009 org/apache/
0 Thu Jan 29 09:55:10 MST 2009 org/apache/hadoop/
0 Thu Jan 29 09:55:10 MST 2009 org/apache/hadoop/hive/
0 Thu Jan 29 09:55:10 MST 2009 org/apache/hadoop/hive/conf/
6447 Thu Jan 29 09:55:10 MST 2009
org/apache/hadoop/hive/conf/HiveConf$ConfVars.class
9123 Thu Jan 29 09:55:10 MST 2009
org/apache/hadoop/hive/conf/HiveConf.class
Thanks,
Shirley
Ashish Thusoo wrote:
> hive_common.jar is there on the classpath. Can you do the following
>
> jar tvf /users/scohen/hive/build/dist/bin/../lib/hive_common.jar
>
> and see if HiveConf shows up in that...
>
> Ashish
>
> -----Original Message-----
> From: Shirley Cohen [mailto:[email protected]]
> Sent: Thursday, January 29, 2009 2:26 PM
> To: [email protected]
> Subject: Re: classpath issue
>
> Great! Here is the print out of my classpath:
>
>
/users/scohen/hive/build/dist/bin/../conf:/users/scohen/hive/build/dist/bin/../lib/TestSerDe.jar:/users/scohen/hive/build/dist/bin/../lib/antlr-runtime-3.0.1.jar:/users/scohen/hive/build/dist/bin/../lib/asm-3.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-cli-2.0-SNAPSHOT.jar:/users/scohen/hive/build/dist/bin/../lib/commons-collections-3.2.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-jexl-1.1.jar:/users/scohen/hive/build/dist/bin/../lib/commons-lang-2.4.jar:/users/scohen/hive/build/dist/bin/../lib/commons-logging-1.0.4.jar:/users/scohen/hive/build/dist/bin/../lib/commons-logging-api-1.0.4.jar:/users/scohen/hive/build/dist/bin/../lib/derby.jar:/users/scohen/hive/build/dist/bin/../lib/hive_anttasks.jar:/users/scohen/hive/build/dist/bin/../lib/hive_cli.jar:/users/scohen/hive/build/dist/bin/../lib/hive_common.jar:/users/scohen/hive/build/dist/bin/../lib/hive_exec.jar:/users/scohen/hive/build/dist/bin/../lib/hive_jdbc.jar:/users/scohen/hive/build/dist/bin/../lib/hive_metastore.jar:/users/scohen/hive/build/dist/bin/../lib/hive_serde.jar:/users/scohen/hive/build/dist/bin/../lib/hive_service.jar:/users/scohen/hive/build/dist/bin/../lib/jdo2-api-2.1.jar:/users/scohen/hive/build/dist/bin/../lib/jline-0.9.94.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-core-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-enhancer-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/jpox-rdbms-1.2.2.jar:/users/scohen/hive/build/dist/bin/../lib/json.jar:/users/scohen/hive/build/dist/bin/../lib/junit-3.8.1.jar:/users/scohen/hive/build/dist/bin/../lib/libfb303.jar:/users/scohen/hive/build/dist/bin/../lib/libthrift.jar:/users/scohen/hive/build/dist/bin/../lib/log4j-1.2.15.jar:/users/scohen/hive/build/dist/bin/../lib/stringtemplate-3.1b1.jar:/users/scohen/hive/build/dist/bin/../lib/velocity-1.5.jar:
>
> Please let me know your thoughts.
>
> Thanks,
>
> Shirley
>
> Prasad Chakka wrote:
> >> You can change the last line bin/hive script to print the
classpath,
>> may we can figure out from that.
>>
>>
>>
----------------------------------------------------------------------
>> --
>> *From: *Shirley Cohen <[email protected]>
>> *Reply-To: *<[email protected]>
>> *Date: *Thu, 29 Jan 2009 11:04:23 -0800
>> *To: *<[email protected]>
>> *Subject: *classpath issue
>>
>> Hi,
>>
>> I'm finally getting around to installing hive. I was able to build
the
>> latest version from the repository without any problems. However,
when
>> I try to run cli from from build/dist/bin, I get the following error:
>>
>> ./bin/hive -hiveconf hive.root.logger=INFO,console
>> java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Unknown Source)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
>> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.hive.conf.HiveConf
>> at java.net.URLClassLoader$1.run(Unknown Source)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(Unknown Source)
>> at java.lang.ClassLoader.loadClass(Unknown Source)
>> at java.lang.ClassLoader.loadClass(Unknown Source)
>> at java.lang.ClassLoader.loadClassInternal(Unknown Source)
>> ... 7 more
>>
>> From the error, it looks as though my classpath isn't getting set
>> correctly. Any ideas on why that might be?
>>
>> I'm running on FedoraCore 8 and hadoop 19. I have set the
$HADOOP_HOME
>> variable to the root of my hadoop directory.
>>
>> Thanks,
>>
>> Shirley
>>
>>
>> >
>
>
