On Wed, Sep 28, 2011 at 6:28 PM, Roman Shaposhnik <[email protected]> wrote: > On Wed, Sep 28, 2011 at 11:54 AM, <[email protected]> wrote: >> This is great progress folks !!! > > Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23 > branch in Bigtop and hooked up 3 additional Jenkins jobs: > http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/ > http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/ > http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/ > > The good news is that HDFS and common are ok, can be packaged and > uses. The bad news for .23 are way more plentiful though: > 1. HBase now breaks with the profile that is supposed to be compatible > with .23. Perhaps this is one of the issues that Todd has mentioned > earlier, but I'm not sure. > > 2. MR2 (or more precise YARN) seem to be in a pretty weird shape. > First of all, I can't really build it the same way that the rest of the > Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt). > Somehow it requires assembly:single goal to be executed explicitly. > Worse yet, it seems to be following a different binary layout compared to > HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how > to make natives bits of pipes/utils compile short of manual execution > of ant. > > 3. fuse-dfs is now an orphan. Its not hooked to the build system and seems > to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757 > > I would really, really, really appreciate if somebody can help triaging these > issues or at least would be nice enough to point me to existing > JIRAs/discussions.
Regarding 2. At the moment you can build MR following the instructions in the INSTALL file. In fact the nightly build set up by Giri and Mahadev builds a MR tarball (https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-MapReduce-trunk). There are some JIRAs to remove the inconsistencies with the rest of the build, such as MAPREDUCE-3024, MAPREDUCE-3014, HADOOP-7590, HADOOP-7642. I hope to work on some of these soon, but any help is always appreciated. Cheers, Tom > > My goal here is to get us to a point where we can start executing > automated integration testing against a fully distributed cluster and > start publishing these results on the same Jenkins to monitor the > health and progress of .22/.23 and corresponding Bigtop stacks. > > Thanks, > Roman. >
