Thank Youngwoo,
I checked out the last Hive build from trunk which contains the Hive1264
patch. I was able to get pass last error message and able to query the
Hbase table. However, when I try to copy the data from Hbase into a Hive
table, I get the following error message. Does anyone know what this error
message is related to and what additional steps I can do to resolve it?
Greatly appreciated
-ray
2010-10-25 04:19:07,406 INFO org.apache.hadoop.util.NativeCodeLoader:
Loaded the native-hadoop library
2010-10-25 04:19:07,579 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: fs.trash.interval; Ignoring.
2010-10-25 04:19:07,581 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
2010-10-25 04:19:07,581 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
mapred.tasktracker.reduce.tasks.maximum; Ignoring.
2010-10-25 04:19:07,582 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: fs.checkpoint.dir; Ignoring.
2010-10-25 04:19:07,585 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
mapred.tasktracker.map.tasks.maximum; Ignoring.
2010-10-25 04:19:07,587 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
hadoop.rpc.socket.factory.class.default; Ignoring.
2010-10-25 04:19:07,589 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.child.ulimit; Ignoring.
2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: tasktracker.http.threads;
Ignoring.
2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.job.tracker.handler.count;
Ignoring.
2010-10-25 04:19:07,592 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2010-10-25 04:19:07,625 INFO org.apache.hadoop.mapred.TaskRunner:
Creating symlink:
/data/data03/hadoop/mapred/local/taskTracker/distcache/-3601043851674017223_220470838_1652071132/hadoop229.qwapi.com/tmp/hive-hadoop/hive_2010-10-24_23-19-13_728_5598498044419457333/-mr-10004/e870a146-897b-428a-b55c-6ae93d8dc47a
<-
/data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./HIVE_PLANe870a146-897b-428a-b55c-6ae93d8dc47a
2010-10-25 04:19:07,637 INFO
org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating
symlink:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/jars/.job.jar.crc
<-
/data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./.job.jar.crc
2010-10-25 04:19:07,643 INFO
org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating
symlink:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/jars/job.jar
<-
/data02/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/work/./job.jar
2010-10-25 04:19:07,654 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2010-10-25 04:19:07,754 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: fs.trash.interval; Ignoring.
2010-10-25 04:19:07,755 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.local.dir; Ignoring.
2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
mapred.tasktracker.reduce.tasks.maximum; Ignoring.
2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.data.dir; Ignoring.
2010-10-25 04:19:07,756 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: fs.checkpoint.dir; Ignoring.
2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.permissions; Ignoring.
2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.block.size; Ignoring.
2010-10-25 04:19:07,758 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
mapred.tasktracker.map.tasks.maximum; Ignoring.
2010-10-25 04:19:07,759 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter:
hadoop.rpc.socket.factory.class.default; Ignoring.
2010-10-25 04:19:07,759 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.namenode.handler.count;
Ignoring.
2010-10-25 04:19:07,760 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.child.ulimit; Ignoring.
2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: tasktracker.http.threads;
Ignoring.
2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: mapred.job.tracker.handler.count;
Ignoring.
2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2010-10-25 04:19:07,762 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.datanode.du.reserved;
Ignoring.
2010-10-25 04:19:07,763 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.name.dir; Ignoring.
2010-10-25 04:19:07,763 WARN org.apache.hadoop.conf.Configuration:
/data/data04/hadoop/mapred/local/taskTracker/hadoop/jobcache/job_201010231249_0013/attempt_201010231249_0013_m_000002_0/job.xml:a
attempt to override final parameter: dfs.datanode.handler.count;
Ignoring.
2010-10-25 04:19:08,054 FATAL org.apache.hadoop.mapred.Child: Error
running child : java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.HTable.<init>(Lorg/apache/hadoop/conf/Configuration;[B)V
at
org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getRecordReader(HiveHBaseTableInputFormat.java:88)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:233)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:363)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:317)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
On Sun, Oct 24, 2010 at 7:39 PM, 김영우 <warwit...@gmail.com> wrote:
> Hi ray,
>
> https://issues.apache.org/jira/browse/HIVE-1264
>
> You should upgrade Hive to 0.7(trunk). CDH3beta3 includes hadoop security
> features.
>
> - Youngwoo
>
> 2010/10/25 Ray Duong <ray.du...@gmail.com>
>
> Hi,
>>
>> I'm getting the following error message after upgrading to CDH3b3. Does
>> anyone know how to resolve this?
>>
>> Thanks
>> -ray
>>
>>
>> hive> show tables;
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/hadoop/security/UnixUserGroupInformation
>>
>> at
>> org.apache.hadoop.hive.ql.processors.CommandProcessorFactory.get(CommandProcessorFactory.java:63)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:133)
>>
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:199)
>>
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:353)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>>
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.security.UnixUserGroupInformation
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
>>
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>
>> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
>>
>> ... 9 more
>>
>
>
