Hi Peter,
I was unable to reproduce your problem using a fresh install
of hadoop-0.20.1+169.68 from CDH2 and hive-0.5.0+20 from CDH3. I also
verified that the hadoop-0.20.1+169.68-core.jar contains
org.apache.hadoop.mapred.TextInputFormat. I agree that this is most likely a
classpath issue. My advice is to start with a fresh install
of hadoop-0.20.1+169.68 and hive-0.5.0+20 and see if you can reproduce the
problem. I also recommend checking to see if you are already setting any
Hadoop or Hive related environment variables that affect classpath and
unsetting these variables if possible.
If none of this works my last suggestion is to modify the bin/hadoop shell
script so that it runs in Bash's debug mode, (e.g. change the first line
"#!/usr/bin/env bash" to "#!/usr/bin/env bash -x") and inspect the debug
output to see if the classpath is getting built correctly.
Thanks.
Carl
On Tue, Sep 14, 2010 at 1:42 PM, Tianqiang Li <peter...@gmail.com> wrote:
> Hi, hive-users,
> I am a new Hive users, install hive recently, when I type any query-related
> command in Hive cli, it throws the exception, but create table are ok:
>
> $ hive
> Hive history file=/tmp/pli/hive_job_log_pli_201009141519_1503313446.txt
> hive> create table test5(a int);
> OK
> Time taken: 2.551 seconds
> hive> show tables;
> OK
> *Failed with exception java.io.IOException:java.io.IOException: Cannot
> create an instance of InputFormat class
> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!*
> Time taken: 0.09 seconds
> hive>
>
> I searched on internet, and there are some info about hive's
> HADOOP_CLASSPATH getting overwritten by some other tools in hadoop-env.sh, I
> tried to use appending( export
> HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:other_paths ), didn't fix the problem,
> then I tried to copy the jar files of hadoop libraries (
> hadoop-0.20.1+169.68-core.jar ) which include *
> org.apache.hadoop.mapred.TextInputFormat,* to $HIVE_HOME/lib/, still the
> problem remain the same, It looks like Hive doesn't know where hadoop's
> libraries are, and failed to new a TextInputFormat object, are there someone
> met this before? Any hints on how to work this around are welcome. Thanks.
>
> Here are some more context, I use hadoop 0.20.1+169.68 from cloudera CDH2,
> and Hive 0.4/0.5 from CDH2/3, (both version for hive have the same issues),
> here is the trace stack in log file.
> ------
> 2010-09-14 15:19:58,826 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.core.resources" but it cannot be resolved.
> 2010-09-14 15:19:58,826 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.core.resources" but it cannot be resolved.
> 2010-09-14 15:19:58,828 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.core.runtime" but it cannot be resolved.
> 2010-09-14 15:19:58,828 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.core.runtime" but it cannot be resolved.
> 2010-09-14 15:19:58,829 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.text" but it cannot be resolved.
> 2010-09-14 15:19:58,829 ERROR DataNucleus.Plugin
> (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
> "org.eclipse.text" but it cannot be resolved.
> 2010-09-14 15:20:04,444 ERROR CliDriver (SessionState.java:printError(279))
> - Failed with exception java.io.IOException:java.io.IOException: Cannot
> create an instance of InputFormat class
> org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
> java.io.IOException: java.io.IOException: Cannot create an instance of
> InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in
> mapredWork!
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:273)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:133)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:440)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:131)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.IOException: Cannot create an instance of InputFormat
> class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getInputFormatFromCache(FetchOperator.java:113)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:249)
>
> ----
> My hadoop-env.sh :
> export
> HADOOP_CLASSPATH_OTHER=/usr/lib/hadoop-0.20/hadoop-0.20.1+169.68-core.jar:/usr/lib/hadoop-0.20/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop-0.20/lib/*:/usr/lib/hadoop-0.20/*
> export HADOOP_CLASSPATH=${HADOOP_CLASSPATH}$:{HADOOP_CLASSPATH_OTHER}
>
>
>
>
> Regards,
> Peter Li*
> *
