Hi, Did you confirm that "org.cloudera.htrace.Trace" was actually added at runtime? As far as I understand, PIG-3285 adds dependency jars only if they're present in classpath on the client side.
You can perhaps turn on DEBUG logs and look for this line<https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java#L790>in the console output of Pig? Thanks, Cheolsoo On Tue, Mar 25, 2014 at 1:04 AM, lulynn_2008 <[email protected]> wrote: > Hi All,I am reading hbase table as following: A = LOAD 'APE1_RATED_EVENT' > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('', '-loadKey true') > AS (id:bytearray); > B = GROUP A BY id; > X = FOREACH B GENERATE COUNT_STAR(A); > DUMP X > > The job failed, and I found following error in hadoop task log. In > PIG-3285, htrace*.jar has been added via "addClassToJobIfExists(job, > "org.cloudera.htrace.Trace");". Any idea why this issue still happened? > Thanks > > > ERROR: > 2014-03-24 23:44:52,090 ERROR > org.apache.hadoop.hbase.mapreduce.TableInputFormat: java.io.IOException: > java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:383) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:360) > at > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:244) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:187) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:149) > at > org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:99) > at > org.apache.pig.backend.hadoop.hbase.HBaseTableInputFormat$HBaseTableIFBuilder.build(HBaseTableInputFormat.java:78) > at > org.apache.pig.backend.hadoop.hbase.HBaseStorage.getInputFormat(HBaseStorage.java:669) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:117) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:487) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:729) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at > java.security.AccessController.doPrivileged(AccessController.java:362) > at javax.security.auth.Subject.doAs(Subject.java:573) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:56) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:39) > at java.lang.reflect.Constructor.newInstance(Constructor.java:527) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:381) > ... 16 more > Caused by: java.lang.NoClassDefFoundError: org.cloudera.htrace.Trace > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:196) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479) > at > org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) > at > org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:794) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:627) > ... 21 more > Caused by: java.lang.ClassNotFoundException: org.cloudera.htrace.Trace > at java.net.URLClassLoader.findClass(URLClassLoader.java:434) > at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:703) > at java.lang.ClassLoader.loadClass(ClassLoader.java:682) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:358) > at java.lang.ClassLoader.loadClass(ClassLoader.java:665) > ... 27 more > > >
