Rakesh, thank you for the links! I agree with Flavio about keeping this a separate project. Having said that, at the point I'm not 100% sure whether the intern will implement ZAB completely from scratch, or start from a fork of the ZooKeeper code base. At this point I'm somewhat leaning towards using the ZooKeeper code base as a starting point. As Ivan pointed out, it's pretty ambitious to implement ZAB correctly in a short amount of time, and it would be good to have something demonstrable at the end of the internship.
On Mon, Jun 2, 2014 at 9:19 AM, FPJ <[email protected]> wrote: > I have a few reasons for suggesting a separate project: > > - I don't see a reason for tying the releases of an independent > implementation of Zab to ZooKeeper > - The set of developers (and committers) interested in an independent > implementation of Zab might be different compared to ZooKeeper; it could > really be a separate community > - It really feels like parallel efforts along the lines of Curator and > BookKeeper, so I see it following similar steps > > Regarding the effort of an intern, I guess it depends how far you want the > initial stretch to go. An initial implementation to contribute to Apache > followed by community activity might get it going. > > -Flavio > >> -----Original Message----- >> From: Ivan Kelly [mailto:[email protected]] >> Sent: 02 June 2014 15:58 >> To: [email protected]; [email protected] >> Cc: Ivan Kelly >> Subject: Re: intern project idea: decouple zab from zookeeper >> >> > 1. BookKeeper is pretty heavyweight, as you need to deploy ZooKeeper >> > and bookies. I think there are use cases where you don't need the >> > horizontal scalability BookKeeper provides, and you prefer to have a >> > light-weight library for replicating state. ZooKeeper is one such >> > example :) >> I was thinking from the point of view that if you want to provide ZAB as a >> library, then the library will have to provide an RPC mechanism for > talking to >> other members of the quorum, and a means to persist updates to disk >> before responding, and _then_ provide a ZAB implementation somewhere >> in between. This doesn't seem much lighter than BK. >> >> I think it's a worthwhile thing to pursue, but I disagree that a separate > project >> is a better way to doing it. If this is an intern project, expecting them > to >> reimplement ZAB might be a bit of a large ask (depending on the internship >> length and the intern themselves). An investigation into splitting the > user >> interface layer of zookeeper and ZAB seems itself to be a nice chunk to > work >> on, and it has the advantage that even if the changes don't get merged > into >> trunk, there will be a clearer picture as to why they can't be split. >> >> > 2. Please correct me if I'm wrong, but BookKeeper is not designed for >> > maintaining multiple in-memory replicas. A ledger can't be opened for >> > reading if it's already open for writing, and you need to recover by >> > restoring from a snapshot and replaying log entries if the writer goes >> > down. >> You can read from a ledger while it is being written to, but right now > it's >> polling. Twitter are working on some changes to make it more notification >> like to reduce latency between the primary writing and the secondary >> reading. >> >> -Ivan > >
