For those people who aren't on the Apache Incubator mailing lists, Tom
White has proposed "BigTop", which is the tooling to integrate the build
and testing of the Apache Hadoop technologies, perhaps eventually to
have coordinated releases.
http://wiki.apache.org/incubator/BigtopProposal
For this to work, it needs support from all the hadoop-related projects
in the ASF, and, in an ideal world, integration with those bits from
outside (JUnit, Jetty) as well as downstream code (Cascading, etc). What
is key is that the projects it builds and tests will need to care about
the test failure reports, because BigTop should not be applying any
patches to private branches of the specific projects (the way Cloudera
and Ubuntu have done with the Hadoop and Linux source trees,
respectively). Some kind of synchronized release schedule would be nice
too, but I view that as a feature creep once the basic set up "build and
test everything" is done.
This could also be the place to put the functional tests against live
clusters, tests which could eventually become the definition of
compatibility that we've discussed before.
For these reasons
-I think everyone in the Hadoop-* projects dev teams ought to get
involved, initially in reviewing the proposal, then hopefully on the
mailing lists.
-It might make sense for the hadoop, pig, hive, hama, mahout, hbase &c
committers to all have access to this, so that they can help evolve the
packaging and functional tests.
thoughts?
-steve
