Niels, Am moving this to hbase-user, since its more relevant to HBase here than MR's typical job submissions.
My reply below: On Tue, May 3, 2011 at 7:12 PM, Niels Basjes <[email protected]> wrote: > Hi, > > I've written my first very simple job that does something with hbase. > > Now when I try to submit my jar in my cluster I get this: > > [nbasjes@master ~/src/catalogloader/run]$ hadoop jar > catalogloader-1.0-SNAPSHOT.jar nl.basjes.catalogloader.Loader > /user/nbasjes/Minicatalog.xml > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/hbase/HBaseConfiguration > at nl.basjes.catalogloader.Loader.main(Loader.java:156) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > ... > > So what is the correct way of doing this? The best way to write a Job Driver for HBase would be to use its TableMapReduceUtil class to make it add dependent jars, prepare jobs with a Scan, etc. [1]. Once your driver reflects the use of TableMapReduceUtil, simply do (assuming HBase's bin/ is on PATH as well): $ HADOOP_CLASSPATH=`hbase classpath` hadoop jar nl.basjes.catalogloader.Loader /user/nbasjes/Minicatalog.xml The "hbase classpath" command is magic for generating the proper hbase-env.sh classpath out for use. If you would still like to use -libjars to add in aux jars, make your Driver use the GenericOptionsParser class [2]. Something like: main(args) { parser = new GenericOptionsParser(args); conf = parser.getConfiguration(); rem_args = parser.getRemainingArgs(); // Do extra args processing if any.. // use 'conf' for your Job, not a new instance. } [1] - http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.html [2] - http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/util/GenericOptionsParser.html HTH :) -- Harsh J
