Re: Zookeeper instead of CLD in Hail
On 06/07/2010 09:21 AM, Jeff Darcy wrote: On 06/04/2010 11:27 PM, Pete Zaitcev wrote: About the focus, ZK is just like CLD from a certain angle (it has the good old files and provides a set of un-posixy operations on them: watches, uniques, "ephemerals"), but it's also entirely unlike CLD (e.g. no locks in the protocol). CLD's model is that clients are daemons, each of which reads a few of its files, maybe locks one or two at boot, and then nothing happens except keepalives. Zookeeper's model... honestly I don't know what it is because it's never explained concisely, but the docs that I saw seem to imply huge numbers of clients all doing random ops all the time on the same files, enough to cause a herd concerns. It looks like Yahoo may be using Zookeeper as a lease manager or something. Crazy. According to the Chubby paper (available at e.g. http://labs.google.com/papers/chubby.html) Chubby itself is mostly used as a naming service. They even say explicitly that few clients hold locks (page 11, left column, under "use and behavior"). I heard people say they cribbed from the same Chubby paper, but it's bollocks. It's absolutely nothing like what Chubby implies. No locks for one thing. I guess it could be argued that locks are the essential feature of Chubby - both the name of the service and the title of the paper refer to it - but I think your dismissal of ZK and Chubby as unrelated is itself bollocks. Watches and ephemerals within a hierarchical namespace of small pseudo-files are also central to how Chubby is actually used within Google, even without locking. ZK copied those features, along with many of the underlying algorithms and protocols. CLD copied a slightly different set. There's still a lot of overlap, even if the one feature that has been overused in Hail is absent (by design) in ZK. I think you're overselling that angle a bit. Google discourages use of Chubby as a strict publish-subscribe mechanism, so watches and ephemerals aren't the bread-and-butter of Chubby necessarily. A lot of the supposed commonality comes simply from the attribute of being a centralized respository of data for autonomous cloud systems -- a shared, highly reliable filesystem -- not any particular attribute related to watches or ephemerals. So I do not see an upside for us to switch at this point, and I have better things to do than learning Zookeeper ropes for weeks. That's a fair enough evaluation. I think I'm the one who might have suggested investigating ZK, but the reasons for that are largely community-related and not technical. If the technical barriers are too high, then it's probably not worth the bother. That does, however, leave the door open for someone else to provide more scalable solutions for the same problems by using more appropriate primitives. If the goal of Hail is to provide building blocks for people to use in constructing their own higher-level service, then it seems to me that the building blocks should be of a familiar shape and the adoption of ZK is about 100x that of CLD for similar purposes. Not sure what you mean by familiar shape? ZK design is too haphazard, and developers IMO have an easier time grasping CLD's fundamental FS-like API. The implementation is also 3x or more in terms of size, compared to CLD. I'd rather get the basics right first, then pursue a popularity contest :) Jeff -- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Zookeeper instead of CLD in Hail
On 06/04/2010 11:27 PM, Pete Zaitcev wrote: I heard people say they cribbed from the same Chubby paper, but it's bollocks. It's absolutely nothing like what Chubby implies. No locks for one thing. To be sure, Zookeeper provides a canned piece of code which implements locks, kinda like you can implement compare-and-swap using Dekker's algorithm on a CPU that doesn't have it. The canned lock creates "sequenced" files (using a ZK server call that creates unique filenames), then sets some "watches" (same as CLD offers), then re-reads the directory to find the lowest number sequential file, which is the winner of the lock. Haha, only serious. I tested it, it works, but ew. Yeah, the main similarity is... both ZK and CLD offer some type of filesystem (with all that implies). ZK is IMO not much like Chubby at all, in terms of focus / design goals. Jeff -- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Zookeeper instead of CLD in Hail
On Mon, 07 Jun 2010 09:21:07 -0400 Jeff Darcy wrote: > > So I do not see > > an upside for us to switch at this point, and I have better things > > to do than learning Zookeeper ropes for weeks. > > That's a fair enough evaluation. I think I'm the one who might have > suggested investigating ZK, but the reasons for that are largely > community-related and not technical. If the technical barriers are too > high, then it's probably not worth the bother. [] I am hoping someone else could do it. Although, "it" includes not just throwing out cldu.c once, but actively maintaining the replacement. Ideally even maintaining an RPM for ZK in Fedora. -- Pete -- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Zookeeper instead of CLD in Hail
On 06/04/2010 11:27 PM, Pete Zaitcev wrote: > About the focus, ZK is just like CLD from a certain angle (it has > the good old files and provides a set of un-posixy operations > on them: watches, uniques, "ephemerals"), but it's also entirely > unlike CLD (e.g. no locks in the protocol). CLD's model is that > clients are daemons, each of which reads a few of its files, maybe > locks one or two at boot, and then nothing happens except keepalives. > Zookeeper's model... honestly I don't know what it is because it's > never explained concisely, but the docs that I saw seem to imply > huge numbers of clients all doing random ops all the time on the > same files, enough to cause a herd concerns. It looks like Yahoo > may be using Zookeeper as a lease manager or something. Crazy. According to the Chubby paper (available at e.g. http://labs.google.com/papers/chubby.html) Chubby itself is mostly used as a naming service. They even say explicitly that few clients hold locks (page 11, left column, under "use and behavior"). > I heard people say they cribbed from the same Chubby paper, but > it's bollocks. It's absolutely nothing like what Chubby implies. > No locks for one thing. I guess it could be argued that locks are the essential feature of Chubby - both the name of the service and the title of the paper refer to it - but I think your dismissal of ZK and Chubby as unrelated is itself bollocks. Watches and ephemerals within a hierarchical namespace of small pseudo-files are also central to how Chubby is actually used within Google, even without locking. ZK copied those features, along with many of the underlying algorithms and protocols. CLD copied a slightly different set. There's still a lot of overlap, even if the one feature that has been overused in Hail is absent (by design) in ZK. > They clearly want daemons to approach the whole problem in a > different way. That's very definitely true. The essential feature here is not synchronization but consensus - not control but data, not "you can't be the leader" but "the leader is X". If what someone needs is locking, and it needs to be done frequently, then I'd say ZK is the wrong tool but I'd also say that such an application is pretty fundamentally non-scalable in ways that have little to do with Chubby/ZK/CLD. People do use ZK for locking, quite successfully, but generally in situations where locking is rare enough to make the inefficiency of layered locking protocols a non-issue (though it still seems they could do better than the protocol you describe). > So I do not see > an upside for us to switch at this point, and I have better things > to do than learning Zookeeper ropes for weeks. That's a fair enough evaluation. I think I'm the one who might have suggested investigating ZK, but the reasons for that are largely community-related and not technical. If the technical barriers are too high, then it's probably not worth the bother. That does, however, leave the door open for someone else to provide more scalable solutions for the same problems by using more appropriate primitives. If the goal of Hail is to provide building blocks for people to use in constructing their own higher-level service, then it seems to me that the building blocks should be of a familiar shape and the adoption of ZK is about 100x that of CLD for similar purposes. -- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
