Yes, I have seen that note and I have added hbase jars to HADOOP_CLASSPATH. May be if I trace through the start scripts of hbase, I might get a clue on why TT is not taking the classpath. But DataNode and HRegionServer added hbase jars to classpath.
Regards, Raghava. On Thu, Jun 10, 2010 at 10:31 AM, Stack <[email protected]> wrote: > Can you figure why its not in the TT CLASSPATH? That the > TableOutputFormat is missing is from the CLASSPATH is what the below > complaint is about. You've seen this note on CLASSPATH and MapReduce? > > http://hbase.apache.org/docs/r0.20.4/api/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath > > St.Ack > > >> > attempt_201005270922_0260_m_000000_0, Status : FAILED > >> > java.lang.RuntimeException: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.hbase.mapreduce.TableOutputFormat > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:808) > >> > at > >> > > org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext. > >> > java:193) > >> > at org.apache.hadoop.mapred.Task.initialize(Task.java:413) > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288) > >> > at org.apache.hadoop.mapred.Child.main(Child.java:170) > .... > > > > On Wed, Jun 9, 2010 at 10:47 PM, Raghava Mutharaju > <[email protected]> wrote: > > Yes, the setup does seem to be identical. The location of hbase jars is > > proper. I checked the process listing, there are hbase jars in > > org.apache.hadoop.hdfs.server.datanode.DataNode and HRegionServer but it > is > > not present in the org.apache.hadoop.mapred.TaskTracker. Is this normal? > > > > Thank you. > > > > Regards, > > Raghava. > > > > On Wed, Jun 9, 2010 at 11:43 AM, Jonathan Gray <[email protected]> > wrote: > > > >> Have you verified that node3 has an identical setup as the other nodes? > >> Did you confirm that all of the hbase jars are in the place pointed to > by > >> HADOOP_CLASSPATH? You can check the process listing of the TaskTracker > or > >> Child processes on node3 to verify exactly where things are pointing and > >> make sure there is an hbase jar listed and that it exists. > >> > >> > -----Original Message----- > >> > From: Raghava Mutharaju [mailto:[email protected]] > >> > Sent: Wednesday, June 09, 2010 3:09 AM > >> > To: [email protected] > >> > Subject: Task attempt fails if assigned to a particular node > >> > > >> > Hello all, > >> > > >> > I am running an MR job and HBase is the data sink. If a task > >> > attempt > >> > is assigned to node3 in the cluster, it fails. Since it failed, it > >> > would get > >> > reassigned to some other node in the cluster and then it succeeds. It > >> > gives > >> > out the following exception on node3. If no task is assigned to node3, > >> > then > >> > no exception gets generated. So it seems to run fine on all the other > >> > nodes > >> > except node3. > >> > > >> > > >> > 10/06/09 05:34:28 INFO mapred.JobClient: Task Id : > >> > attempt_201005270922_0260_m_000000_0, Status : FAILED > >> > java.lang.RuntimeException: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.hbase.mapreduce.TableOutputFormat > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:808) > >> > at > >> > > org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext. > >> > java:193) > >> > at org.apache.hadoop.mapred.Task.initialize(Task.java:413) > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288) > >> > at org.apache.hadoop.mapred.Child.main(Child.java:170) > >> > Caused by: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.hbase.mapreduce.TableOutputFormat > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > >> > at java.security.AccessController.doPrivileged(Native Method) > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > >> > at > >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) > >> > at > >> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > >> > at java.lang.Class.forName0(Native Method) > >> > at java.lang.Class.forName(Class.java:247) > >> > at > >> > > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java: > >> > 761) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:806) > >> > ... 4 more > >> > > >> > > >> > On node3, in hadoop-env.sh, $HADOOP_CLASSPATH has been set to include > >> > hbase > >> > jars and $HBASE_HOME/conf. I suspected this might be the issue, may be > >> > it > >> > wasn't set. But that is not the issue. What could be the reason behind > >> > this? > >> > > >> > Thank you. > >> > > >> > Regards, > >> > Raghava. > >> > > >
