Thank Youngwoo, I checked out the last Hive build from trunk which contains the Hive1264 patch. I was able to get pass last error message and able to query the Hbase table. However, when I try to copy the data from Hbase into a Hive table, I get the following error message. Does anyone know what this error message is related to and what additional steps I can do to resolve it?
Greatly appreciated -ray 2010-10-25 04:19:07,406 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2010-10-25 04:19:07,579 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: fs.trash.interval; Ignoring. 2010-10-25 04:19:07,581 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. 2010-10-25 04:19:07,581 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.tasktracker.reduce.tasks.maximum; Ignoring. 2010-10-25 04:19:07,582 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: fs.checkpoint.dir; Ignoring. 2010-10-25 04:19:07,585 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.tasktracker.map.tasks.maximum; Ignoring. 2010-10-25 04:19:07,587 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: hadoop.rpc.socket.factory.class.default; Ignoring. 2010-10-25 04:19:07,589 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.child.ulimit; Ignoring. 2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: tasktracker.http.threads; Ignoring. 2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.job.tracker.handler.count; Ignoring. 2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2010-10-25 04:19:07,625 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /data/data03/hadoop/mapred/local/taskTracker/distcache/-3601043851674017223_220470838_1652071132/hadoop229.qwapi.com/tmp/hive-hadoop/hive_2010-10-24_23-19-13_728_5598498044419457333/-mr-10004/e870a146-897b-428a-b55c-6ae93d8dc47a <- /data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./HIVE_PLANe870a146-897b-428a-b55c-6ae93d8dc47a 2010-10-25 04:19:07,637 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/jars/.job.jar.crc <- /data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./.job.jar.crc 2010-10-25 04:19:07,643 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/jars/job.jar <- /data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./job.jar 2010-10-25 04:19:07,654 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2010-10-25 04:19:07,754 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: fs.trash.interval; Ignoring. 2010-10-25 04:19:07,755 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.local.dir; Ignoring. 2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.tasktracker.reduce.tasks.maximum; Ignoring. 2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.data.dir; Ignoring. 2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: fs.checkpoint.dir; Ignoring. 2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.permissions; Ignoring. 2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.block.size; Ignoring. 2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.tasktracker.map.tasks.maximum; Ignoring. 2010-10-25 04:19:07,759 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: hadoop.rpc.socket.factory.class.default; Ignoring. 2010-10-25 04:19:07,759 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.namenode.handler.count; Ignoring. 2010-10-25 04:19:07,760 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.child.ulimit; Ignoring. 2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: tasktracker.http.threads; Ignoring. 2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: mapred.job.tracker.handler.count; Ignoring. 2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring. 2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.datanode.du.reserved; Ignoring. 2010-10-25 04:19:07,763 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.name.dir; Ignoring. 2010-10-25 04:19:07,763 WARN org.apache.hadoop.conf.Configuration: /data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a attempt to override final parameter: dfs.datanode.handler.count; Ignoring. 2010-10-25 04:19:08,054 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.HTable.<init>(Lorg/apache/hadoop/conf/Configuration;[B)V at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getRecordReader(HiveHBaseTableInputFormat.java:88) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:233) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:317) at org.apache.hadoop.mapred.Child$4.run(Child.java:217) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) at org.apache.hadoop.mapred.Child.main(Child.java:211) On Sun, Oct 24, 2010 at 7:39 PM, 김영우 <warwit...@gmail.com> wrote: > Hi ray, > > https://issues.apache.org/jira/browse/HIVE-1264 > > You should upgrade Hive to 0.7(trunk). CDH3beta3 includes hadoop security > features. > > - Youngwoo > > 2010/10/25 Ray Duong <ray.du...@gmail.com> > > Hi, >> >> I'm getting the following error message after upgrading to CDH3b3. Does >> anyone know how to resolve this? >> >> Thanks >> -ray >> >> >> hive> show tables; >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/hadoop/security/UnixUserGroupInformation >> >> at >> org.apache.hadoop.hive.ql.processors.CommandProcessorFactory.get(CommandProcessorFactory.java:63) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:133) >> >> at >> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199) >> >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> >> at java.lang.reflect.Method.invoke(Method.java:597) >> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:186) >> >> Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.security.UnixUserGroupInformation >> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> >> at java.security.AccessController.doPrivileged(Native Method) >> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:303) >> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> >> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316) >> >> ... 9 more >> > >