Hi Massoud, Great work!
Yes this is exactly the use of shims. When we see an API change across hadoop versions, we add a new function to shims interface, and implement it in each of the shim. For this one, you probably want to wrap the logic in Driver.java into a single shim interface function, and implement that function in all shim versions. Does that make sense? Zheng On Mon, Mar 1, 2010 at 1:08 PM, Massoud Mazar <massoud.ma...@avg.com> wrote: > Zheng, > > Thanks for answering. > I've decided to give it (hive 0.50 on hadoop 0.22) a try. I'm a developer, > but not a Java developer, so with some initial help I can spend time and work > on this. > Just to start, I modified the ShimLoader.java and copied the same > HADOOP_SHIM_CLASSES and JETTY_SHIM_CLASSES from 0.20 to 0.22 to see where it > breaks. > > I built and deployed hive 0.50 to a running hadoop 0.22 and did "show > tables;" in hive, and I got this: > > Exception in thread "main" java.lang.NoSuchMethodError: > org.apache.hadoop.security.UserGroupInformation: method <init>()V not found > at > org.apache.hadoop.security.UnixUserGroupInformation.<init>(UnixUserGroupInformation.java:69) > at > org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:271) > at > org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:300) > at org.apache.hadoop.hive.ql.Driver.<init>(Driver.java:243) > at > org.apache.hadoop.hive.ql.processors.CommandProcessorFactory.get(CommandProcessorFactory.java:40) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:116) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:187) > > Now, when I look at the UserGroupInformation class in hadoop 0.22 source > code, it does not have a parameter-less constructor, but documentation at > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/security/UserGroupInformation.html > shows such a constructor. > > Now, my question is: is this something that can be fixed by shims? Or it is a > problem with hadoop? > > -----Original Message----- > From: Zheng Shao [mailto:zsh...@gmail.com] > Sent: Saturday, February 27, 2010 4:24 AM > To: hive-user@hadoop.apache.org > Subject: Re: hive 0.50 on hadoop 0.22 > > Hi Mazar, > > We have not tried Hive on Hadoop higher than 0.20 yet. > > However, Hive has the shim infrastructure which makes it easy to port > to new Hadoop versions. > Please see the shim directory inside Hive. > > Zheng > > On Fri, Feb 26, 2010 at 1:59 PM, Massoud Mazar <massoud.ma...@avg.com> wrote: >> Is it possible to run release-0.5.0-rc0 on top of hadoop 0.22.0 (trunk)? >> > > > > -- > Yours, > Zheng > -- Yours, Zheng
