With 0.2 i believe you need to add the jars to your MR job. If you are
using the hadoop command to run your MR job, add -libjars and point to
hcat. If you are not then you should be able to do something like this:
job = Job
job.addFileToClassPath(Path("/path/to/hcat/jar"))
I forget if this is the right way to do it. It should be something like
that. If you use pig, pig should set this up for you.
On Wed, Mar 14, 2012 at 9:18 PM, Monnand <[email protected]> wrote:
> Dear All,
>
> I wrote a MapReduce job using HCatalog, and got the following messages
> when running the Mapper:
>
> 12/03/14 22:04:05 INFO mapred.JobClient: Task Id :
> attempt_201203142145_0007_m_000000_0, Status : FAILED
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hcatalog.mapreduce.HCatInputFormat
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
> at
> org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:723)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hcatalog.mapreduce.HCatInputFormat
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
> ... 8 more
>
> We have already made the following changes and still get the message above:
>
> - copy hcatalog-0.2.0*.jar to ${HADOOP_HOME}/lib
> - run the hadoop with
>
> -Djava.ext.dir="/usr/local/hcat/:/usr/local/hcat/share/hcatalog/:/usr/local/hcat/share/hcatalog/lib/"
> - We also tried -libjars, HADOOP_CLASSPATH but every try is failed.
>
> Can anyone help us to figure out what's going on here?
>
> Regards,
> -Nan
>
