I have a patch to support hadoop2 here https://issues.apache.org/jira/browse/FLUME-1618 Its blocked on the availability of an HBase version compiled against Hadoop2. It appears the next release of HBase will publish both hadoop1 & 2 versions. If you want to get this going locally.. - compile HBase with hadoop 2 profile - grab this patch for flume, revise the HBase version to whatever you built and build Flume using the hadoop-2.0 profile.
-Roshan On Wed, Oct 17, 2012 at 9:42 AM, Brock Noland <[email protected]> wrote: > It looks to me like the first issue is the pom, as Roman suggests: > > $ mvn install -DskipTests -Dhadoop.profile=23 > [INFO] Scanning for projects... > [ERROR] The build could not read 1 project -> [Help 1] > [ERROR] > [ERROR] The project > org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:1.3.0 > > (/Users/noland/workspaces/flume-apache-trunk-golden/flume/flume-ng-sinks/flume-ng-hbase-sink/pom.xml) > has 1 error > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:hadoop-test:jar is missing. @ line 93, column 17 > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with > the -e switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException > > > On Tue, Oct 16, 2012 at 7:01 PM, Jarek Jarcec Cecho <[email protected]> > wrote: > > Sqoop is also having Hbase unit tests disabled for hadoop-2 profile and > everything seems to be working for us. > > > > Jarcec > > > > On Tue, Oct 16, 2012 at 04:58:31PM -0700, Hari Shreedharan wrote: > >> Am I correct in assuming that disabling the Hbase sink unit tests will > fix the issue? That is, even if the Hbase artifacts used in the build is > built against hadoop-1, it would work fine against hbase built against > hadoop-2 (since hbase did not change any interfaces to abstract classes, > hadoop did right?). > >> > >> > >> Thanks, > >> Hari > >> > >> -- > >> Hari Shreedharan > >> > >> > >> On Tuesday, October 16, 2012 at 4:56 PM, Hari Shreedharan wrote: > >> > >> > Hi Roman, > >> > > >> > Even we would like to make sure we can build against hadoop-2. Do you > have suggestions on what we can do to fix this so that you can have > flume-1.3.0 in the next Bigtop release? > >> > > >> > > >> > Thanks, > >> > Hari > >> > > >> > -- > >> > Hari Shreedharan > >> > > >> > > >> > On Tuesday, October 16, 2012 at 4:16 PM, Roman Shaposhnik wrote: > >> > > >> > > Guys, > >> > > > >> > > is there any chance that the issue with hadoop-0.23 > >> > > profile being broken can be fixed in 1.3.0? I'm planning > >> > > a Bigtop 0.5.0 release (due in a couple of month) and > >> > > it would be very unfortunate if we have to stick with > >> > > Flume 1.1.0. > >> > > > >> > > Please let me know if you need any help is solving this. > >> > > > >> > > Please also note, that this is simply an issue of proper > >> > > names in POMs not lack of HBase artifacts. To Bigtop > >> > > (or anybody else for that matter) the lack of HBase > >> > > artifacts compiled against hadoop 2 is simple matter > >> > > of actually building HBase before we build Flume > >> > > and thus guaranteeing local copies of things that are > >> > > correct in our .m2. > >> > > > >> > > Thanks, > >> > > Roman. > >> > > > >> > > > >> > > > >> > > >> > > >> > > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ >
