Greetings, I thought I'd drop a note to update folks on progress of hadoop-0.23.
Things are have been very busy in hadoop-0.23 land. We continue to crank through the issues and get ready to ship. We are mostly pass the initial teething pains of moving our entire build infrastructure to Maven - many thanks to Alejandro, Tom, Giri & Eric Yang. HDFS is nearly there: # HDFS Federation and Client side mount tables have been tested with ~300 node clusters with security turned on. # HDFS upgrades have been tested from 0.20.2xx. # Functional tests for HDFS are complete. NextGen MapReduce (aka MRv2, aka YARN) is coming along great: # We are happy to report we've done extensive scale testing to confirm stability - Sort/GridMixv3 etc. at ~350nodes - Scale testing with simulated clusters of ~1500 nodes # Functional tests for all of MapReduce functionality # Pig (0.9 & 0.9.1) working with NextGen MapReduce # All above have been done with no regressions in security. We are about to finish performance certification for both HDFS & MapReduce in the next couple of weeks too, after which we start integration tests with HBase, Hive, Oozie etc. We have cranked through 75 bugs in September alone (http://s.apache.org/mr-sept) and have another 50-ish bugs to go... we have at least 4 different organizations contributing patches to MRv2 in Sept alone: Yahoo, Hortonworks, LinkedIn & Huawei. Given where we are I'm confident we can have a strong hadoop-0.23.0 release by late October. The current plan is to deploy to alpha clusters in November. Citius, Altius, Fortius! :) Thanks to everyone who contributed, look forward to continued help. Arun PS: I'll continue to provide a periodic updates as we get closer to a hadoop-0.23.0 release.
