Otis Gospodnetic wrote:
Didn't we see Hadoop do exactly the opposite? Doug described it as code-base being too big, but I'd say the original list was also too high traffic (was split into common-, hdfs-, and mapreduce- I believe).
The intent is that eventually HDFS and MapReduce will evolve independently. If a new MapReduce feature depends on a new HDFS feature, then MapReduce would have to wait until HDFS releases before it can release. Over time the expectation is that this won't happen much. MapReduce currently develops against a nightly "snapshot" build of HDFS and releases will be synchronized for a while yet, but it may someday switch to developing against released versions of HDFS. So the direction there is the opposite of what's been proposed here: introducing layered project dependencies rather than reducing them. It's too soon to say how well that split will work, just as I think it's difficult to guess how well merging Lucene and Solr will fare. From experience, we know the downsides of Lucene and Solr being split, now we may have the opportunity to learn the downsides of being merged!
Doug
