Since I've been asked to make the comparison a few times, it's become clearer what I think the difference between scuttlebutt and gossipmonger is. It actually starts to feel like comparing apples to oranges.
scuttlebutt: "peer-to-peer replicatable data structure", two (maybe more?) scuttlebutts connect, negotiate the differences, and synchronize gossipmonger: endpoint that uses connection-less gossip protocol to find other gossipmongers, track whether they are live or not, and use "scuttlebutt reconciliation" strategy to disseminate information to other gossipmongers. Perhaps the above comparison is shorter and more useful :) There are other projects I'm working on that use gossipmonger as a component. When those become easier to consume, they might better illustrate the reason for gossipmonger's existence. Cheers! On Sunday, October 20, 2013 6:07:07 AM UTC-5, Floby wrote: > > Thanks for taking the time to write a detailed reply. It's much > appreciated. > > My point of view is that I'm not aware of all the details of the science > under the hood and I'm pretty happy this way for now. Dominic's > implementation provides a very small API surface which is in turn very > simple to learn (instanciate it, pipe it, use it). Extending his > implemntation is also much simpler and requires less knowledge of the > implementation details (only 2 methods to override). I haven't taken a > closer look at your module yet but it already seems to me that I have to > learn a whole lot of new concepts that I don't really need to know for a > basic use. > > My primary use case at the moment is synchronizing client side models (in > a browser) accross multiple users. I mostly had to combine scuttlebutt, > shoe and mux-demux. As you see, I'm only dealing with connected transports > (websockets, http) exposed as streams. The stream approach to scuttlebutt > makes it really easy to compose with other modules in my opinion. > > I'll make sure to give a try to your implementation for another project of > mine =) > > On Friday, 18 October 2013 16:06:20 UTC+2, Tristan Slominski wrote: >> >> Hey Kevin and Floby, >> >> Heh.. I have a hard time of thinking of a short answer to this, so please >> be patient. >> >> I wanted to write an open source gossipmonger for a while now. My initial >> implementation of gossip was closed source and at the time dominictarr's >> version didn't exist. That's not an argument for primacy or anything, I >> just want to highlight that I've been living with this implementation >> approach in my head for a long time, so my intuition has been indoctrinated >> in the approach demonstrated in gossipmonger. I was excited to see >> dominictarr's work (and substack's, as well as others in the community) as >> they shared their vision for peer-to-peer designs. So, when I needed an >> implementation for my current project, naturally I took a look at >> scuttlebutt, but overriding any reasons I can highlight below, I already >> had something that I knew worked, so I didn't really have too much >> incentive to learn something else that also worked :D. I'm as lazy as the >> next human. >> >> Having said that, I also don't know your use cases, so it's difficult to >> compare anything in the dark. All I can offer are some initial observations >> for what felt different in scuttlebutt compared to the gossipmonger >> implementation that I was familiar with for my particular use case. >> >> My use case/criteria for gossip are: >> >> 1. Promoting global awareness of peer-state. That is, given peer1, peer2, >> and peer3, only peer1 writes peer1 state, only peer2 writes peer2 state, >> and only peer3 writes peer3 state. However, I want peer2 to know the state >> of both peer1 and peer3, and so on. >> 2. Liveness determination (which is pretty much "phi accrual failure >> detection"). >> 3. Asynchronous communication. Specifically, I mean that I don't want to >> force a request-reply pattern of interaction. Even more specifically :), I >> don't want to force a connection-oriented pattern of interaction. You could >> also call this a "push-only" requirement. Part of the reason for this is >> that my initial gossip implementation was implemented over a messaging >> service (initial prototype was actually over pubnub ( >> http://www.pubnub.com/), although I moved away from that later, which >> ties into the next point). >> 4. Transport independence. That is, I wanted to be able to have an API >> that I could implement using messages in a bottle if I wanted to. >> >> So... >> >> "Scuttlebutt is always duplex. Scuttlebutt does a handshake..." ( >> https://github.com/dominictarr/scuttlebutt#gotchas). That doesn't meet >> my criteria #3. This also seems to me that it won't work over UDP. UDP >> implementation is hinted at in Introduction section of "Efficient >> Reconciliation and Flow Control for Anti-Entropy Protocols" ( >> http://www.cs.cornell.edu/home/rvr/papers/flowgossip.pdf): "Gossip >> protocols are designed to be non-invasive and have predictable performance, >> and for this a designer has to fix not only the gossip rate per participant >> but also the maximum size of gossip messages (e.g., maximum UDP packet >> size)." >> >> Another factor... >> >> "i.each(self.history(sources), function (data) {d._data(data)})" ( >> https://github.com/dominictarr/scuttlebutt/blob/8217ec7f96091838be3b56122d16176ba2b63fa6/index.js#L150). >> >> This indicates to me that the entire history since the last timestamp is >> sent over the wire (I may be wrong in this, but it is the impression that I >> got). Taking look at section 3.2, Scuttlebutt Reconciliation, of "Efficient >> Reconciliation and Flow Control >> for Anti-Entropy Protocols" ( >> http://www.cs.cornell.edu/home/rvr/papers/flowgossip.pdf) the authors >> would call this "precise reconciliation", which can result in sending >> updates that are unnecessary between the peers, "scuttlebutt >> reconciliation", on the other hand, converges more slowly, but "leaves room >> in the gossip message for deltas from other participants." Another >> consideration in that section is how to pick which deltas are sent in a >> gossip message. The authors outline "scuttle-breadth" and "scuttle-depth" >> approaches and say that "scuttle-depth" seemed to work better. I haven't >> been able to figure out where in the source code of scuttlebutt these >> things come into play (gossipmonger uses the "scuttle-depth" approach). To >> be fair, there may be subtleties in the implementation that I'm not seeing, >> but when in doubt I already had my implementation that met my criteria :). >> You could say I time-boxed my analysis of the source code. >> >> To summarize, for my four criteria, #2 was missing (or external), #3 >> seemed to not be satisfied, and #4 seemed to not be satisfied based on #3 >> not being satisfied. 3 out of 4 were missing, which was enough for me to >> open source an implementation I already had on hand. >> >> I hope this helps, and if my analysis was incorrect, then great, I'd love >> to have someone walk me through how my use case could be implemented in >> scuttlebutt. Scuttlebutt is great, and it had much more eyeballs for much >> longer on it. To be fair, I didn't ask dominictarr about any of this, but >> this goes again back to the fact that I already had an implementation, so >> it was easier/faster to release it. >> >> Cheers, >> >> Tristan >> >> On Friday, October 18, 2013 3:29:59 AM UTC-5, Floby wrote: >>> >>> Same as Kevin, >>> >>> I'm currently using Scuttlebutt and it's working great for my use case. >>> Can you point out the major difference or enhancements ? >>> >>> On Friday, 18 October 2013 02:22:07 UTC+2, Tristan Slominski wrote: >>>> >>>> Hello, >>>> >>>> I'm happy to announce the initial release of Gossipmonger ( >>>> https://github.com/tristanls/gossipmonger). >>>> >>>> Gossipmonger is an implementation of the Scuttlebutt gossip protocol >>>> for real-time peer-to-peer state distribution (and liveness monitoring). >>>> Additionally, it has pluggable storage and transport mechanisms (currently >>>> it ships with in-memory storage and TCP transport), so it is open to be >>>> implemented in the browser as well as server-side. >>>> >>>> Cheers, >>>> >>>> Tristan >>>> >>> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
