Just closing the loop here, it might help someone else to hand patch their build process before I get the patches in the hadoop branch, no changes required for hbase.
I backported the latest version of KerberosUtil from hadoop 2.0 branch and recompiled hadoop-common/hadoop-auth and then installed the jar out to my local maven repository. Ran the command "mvn clean package -Dhadoop.profile=23" and I was able to build hbase against hadoop 0.23. Now starts the more painful part of making sure everything works during runtime :-) Stack, I noticed that in all profiles except 0.23 there is hadoop-core or hadoop-common includes, while in 0.23 there is only hadoop-client as a dependency and there is no mention for hadoop-common or hadoop-auth anywhere, do they get pulled in due to other dependencies ? Just trying to understand the whole build process for hbase. Thanks, Viral On Mon, Jan 28, 2013 at 5:58 PM, Viral Bajaria <[email protected]>wrote: > Tried all of it, I think I will have to defer this to the hadoop mailing > list because it seems there is a missing class in hadoop 0.23 branches, not > sure if that is intentional. The class exists in trunk and hadoop 2.0 > branches. Though the surprising part is that it does not exist in < 0.23. > Does the hbase code base exclude certain files based on what hadoop profile > we are targeting ? > > I will try to apply a patch of that class to 0.23.5 and see if it works > and if it does then I will post it to the list and submit a patch to both > hadoop and hbase (needs pom change). > > -Viral > > > On Mon, Jan 28, 2013 at 5:34 PM, Stack <[email protected]> wrote: > >> The below seems like a good suggestion by Vandana. >> >> I will say that focus is on support for hadoop 1 and 2. There has not >> been much call for us to support 0.23.x If you can figure what needs >> fixing, we could try adding the fix to 0.94 (In trunk a patch to add a >> compatibility module for hadoop-0.23.x would be welcomed). >> >> St.Ack >> >> >> On Mon, Jan 28, 2013 at 5:09 PM, Vandana Ayyalasomayajula < >> [email protected]> wrote: >> >> > May be thats the issue. Try downloading the source from 0.94 branch and >> > use the maven command with -Psecurity and -Dhadoop.profile=23. >> > That should work. >> > >> > Thanks >> > Vandana >> > >> > On Jan 28, 2013, at 11:48 AM, Viral Bajaria wrote: >> > >> > > Thanks Vandana for reply. I tried that but no luck. It still throws >> the >> > > same error. I thought there might have been a typo and you meant -D >> and >> > not >> > > -P but none of them worked. >> > > >> > > I verified that the hadoop-auth code base does not have KerberosUtil >> > class >> > > anymore. So I am guessing there is some, but I am surprised no one has >> > > raised this point till now because on the list a few of them say that >> > they >> > > are using hbase 0.94 with hadoop 0.23 >> > > >> > > Maybe I am doing something totally wrong and downloading via the >> > > "downloads" link and compiling is not the right thing to do and I >> should >> > > just get it from the source repository ? >> > > >> > > -Viral >> > > >> > > On Mon, Jan 28, 2013 at 7:43 AM, Vandana Ayyalasomayajula < >> > > [email protected]> wrote: >> > > >> > >> Hi viral, >> > >> >> > >> Try adding "-Psecurity" and then compiling. >> > >> >> > >> Thanks >> > >> Vandana >> > >> >> > >> Sent from my iPhone >> > >> >> > >> On Jan 28, 2013, at 3:05 AM, "Viral Bajaria" < >> [email protected]> >> > >> wrote: >> > >> >> > >>> Hi, >> > >>> >> > >>> Is anyone running hbase 0.94.4 against hadoop 0.23.5 ? If yes, how >> did >> > >> you >> > >>> end up compiling hbase for hadoop 0.23 ? >> > >>> >> > >>> I downloaded the hbase release and tried running "mvn clean package >> > >>> -Dhadoop.profile=23" but I keep on getting a compilation error as >> > >> follows: >> > >>> >> > >>> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ >> > hbase >> > >> --- >> > >>> [INFO] Compiling 738 source files to >> > >>> /home/viral/dev/downloads/hbase-0.94.4/target/classes >> > >>> [INFO] ------------------------------------------------------------- >> > >>> [ERROR] COMPILATION ERROR : >> > >>> [INFO] ------------------------------------------------------------- >> > >>> [ERROR] >> > >>> >> > >> >> > >> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java:[41,53] >> > >>> cannot find symbol >> > >>> symbol : class KerberosUtil >> > >>> location: package org.apache.hadoop.security.authentication.util >> > >>> [ERROR] >> > >>> >> > >> >> > >> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/util/Bytes.java:[45,15] >> > >>> sun.misc.Unsafe is Sun proprietary API and may be removed in a >> future >> > >>> release >> > >>> [ERROR] >> > >>> >> > >> >> > >> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/util/Bytes.java:[1029,19] >> > >>> sun.misc.Unsafe is Sun proprietary API and may be removed in a >> future >> > >>> release >> > >>> [ERROR] >> > >>> >> > >> >> > >> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java:[242,32] >> > >>> cannot find symbol >> > >>> symbol : variable KerberosUtil >> > >>> location: class >> > >> org.apache.hadoop.hbase.zookeeper.ZKUtil.JaasConfiguration >> > >>> >> > >>> Tried with both the security release and the one without but no >> luck. >> > >>> >> > >>> Any pointers ? >> > >>> >> > >>> Thanks, >> > >>> Viral >> > >> >> > >> > >> > >
