What I mean to say, is that: The zk web site mentions recipes.The lock recipe was added into the source tree. The ZK-lock recipe is not heavily tested. I got burned by this code, assuming it was a higher quality than it was. https://issues.apache.org/jira/browse/ZOOKEEPER-1262, starvation, and other flaws. I eventually had to switch my code to use http://curator.apache.org/curator-recipes/.
I am not attempting to bad mouth ZK or the lock recipe, but I want to draw a clear line as to what the core functionality of gossip is. If we introduce recipes or systems such as lock systems or leader election they will need to be built as core functionality. In the next 6 months my target for Gossip is to be able to form peer-to-peer networks between 1-1000 systems. Wire compatible implementations in at least two languages (Java and one other maybe c or python). I think a big want is to be able to gossip custom data. I believe that opens the project up to be middle ware for a lot of applications that need to share data but do not need atomic or transaction guarantees of a system like ZK or etcd. This is a use case of mine: I have small amounts of shared data. I do not believe the requirements justify a 3 node ZK setup. The application instances should be able to share state/data without need to have a leader. They each share data and the data space is keyed by node id. On Thu, Jun 2, 2016 at 5:12 PM, Flavio Junqueira <[email protected]> wrote: > Thanks for the feedback, and again, I'm just curious and trying to > understand what this project is trying to achieve for now. It is clear that > it can change its course. > > I'm not really really sure what you meant to say with that pointer to the > stack overflow question on the zk lock recipe. What's proof of concept > there? I don't follow closely stack overflow questions, but I'm aware of > similar discussions. Perhaps you'd be interested in this reference too: > > https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html > > -Flavio > > > On 02 Jun 2016, at 17:22, Edward Capriolo <[email protected]> wrote: > > > > Good question. We addressed this in our proposal but there is a need to > > expand on it. IMHO we to draw a clear line to say Gossip is not a > database, > > Gossip is not a leader election system. > > > > We are discussing allowing the systems in the cluster to Gossip "custom" > > data. With that it will be a natural progression to allowing someone to > > build something like a leader election on top of it. However, we want to > > draw a careful line when we get to this point. The last thing I want is a > > 'crusty' lock system that does not really work a d > > https://aphyr.com/posts/313-strong-consistency-models tears it to > shreds, > > or one that is more a proof of concept and can be used in production: > > > http://stackoverflow.com/questions/14275613/concerns-about-zookeepers-lock-recipe > > . > > > > However Apache projects should allow people to 'scratch itches'. If we > as a > > group want to take on leader election we will move in that direction, > > however recipe/feature needs to be clean, operate at scale, and not be > > something that is more of a liability than an asset. For example is the > > lock "optimistic/pessimistic"? Does it function in a split brain? and we > > need testable conclusions that we can validate in our release process. It > > can't be the academic lock recipe, that kinda works, but not with and > even > > number of nodes, has starvation issues at scale, and sometimes two or > three > > nodes have the lock etc :) > > > > On Wed, Jun 1, 2016 at 7:33 AM, Flavio Junqueira <[email protected]> wrote: > > > >> Hello there! > >> > >> I was reading the Gossip proposal and was wondering about its use cases. > >> In particular, it argues that it has some overlap with coordination > systems > >> like ZooKeeper, so I'd be curious to hear the thoughts here on what the > >> specific uses cases are and I was hoping to get some clarity on where > it is > >> a good replacement for coordination systems like zk, etcd, and others. > >> > >> The documentation mentions node discovery and cluster liveness. Are you > >> targeting environments in which the latency of propagating new node > events > >> or crashed events doesn't matter? I'm also wondering if it is also part > of > >> the motivation to use that elect a leader or implemented a distributed > lock > >> and such. > >> > >> I hope you folks don't mind me starting this thread as I'm really > curious > >> to learn more about where you see this project going. > >> > >> Thanks, > >> -Flavio > >> > >> > >> > >
