+1. Ben, thank you for your proposal. --Michi
On 3/18/11 2:11 PM, "Benjamin Reed" <[email protected]> wrote: > +1 i'm all for it of course :) > > On Fri, Mar 18, 2011 at 2:11 PM, Benjamin Reed <[email protected]> wrote: >> Proposal >> >> BookKeeper is a distributed write ahead logging (WAL) service. It is >> built on top of ZooKeeper and is used for distributed recovery and >> reliability. Much like ZooKeeper itself, BookKeeper is a distributed >> tool used for reliability, but unlike ZooKeeper it is used to store >> large amounts of application data in the form of byte streams, which >> we call ledgers. It is made up of Bookies, which store data, and a >> client library. All other meta-data is stored in ZooKeeper. >> >> The BookKeeper subproject also includes Hedwig, which is a pub/sub >> system built on both BookKeeper and ZooKeeper. It's coupling with >> BookKeeper is tight and many of the performance features of BookKeeper >> were added in response to Hedwig's requirements. Hedwig is made up of >> a rather thin client library and stateless Brokers that cache and >> distribute messages. >> >> Background >> >> BookKeeper was developed as a WAL for the Hadoop NameNode and was also >> used to build the Hedwig pub/sub system. Both are currently contribs >> to ZooKeeper. The work to get the hooks necessary to integrate >> BookKeeper with the NameNode is almost complete (HDFS-1580). >> >> Rational >> >> We have contributors that we would like to make committers to >> BookKeeper and Hedwig. It would be nice to allow a development >> community to grow around BookKeeper. >> >> Also, hudson does not run against contrib. Making BookKeeper its own >> subproject would allow us to better qa our changes. >> >> We also would like to decouple BookKeeper releases from ZooKeeper >> releases. ZooKeeper is quite mature and has relatively long release >> cycles. We would like shorter release cycles for BookKeeper. >> >> In theory we could make two projects BookKeeper and Hedwig, but doing >> so would double the project management and release overhead. The >> development community between BookKeeper and Hedwig overlaps heavily, >> so we would be increasing the burden on the same group of >> contributors. >> >> Because of the developer community overlap with ZooKeeper and the fact >> that BookKeeper is inline with the general mission of ZooKeeper, we >> think BookKeeper should be a subproject of ZooKeeper. >> >> Call for vote >> >> I propose that BookKeeper become a ZooKeeper subproject subject to >> ZooKeeper PMC and Bylaws. I, Benjamin Reed, will champion the >> proposal. BookKeeper will have the following initial committers: >> >> Dhruba Borthakur (Facebook) >> Flavio Junqueira (Yahoo) >> Ivan Kelly (Yahoo) >> Benjamin Reed (Yahoo) >> Utkarsh Srivastava (Twitter) >> >
