I just realised now you also wrote `discover` which totally fits in the picture you're describing.
For me the most notable difference in use is that gossipmonger is not connected whereas scuttlebutt is. That's why scuttlebutt can use the stream interface and gossipmonger cannot. On Sunday, 20 October 2013 21:22:18 UTC+2, Tristan Slominski wrote: > > Great :) > > It's not easy to consume, but if you're feeling brave, here's one use of > gossipmonger by a coordinator module that decides which machines to > replicate data writes to (it's sort of like dynamo but without consistent > hashing for this particular use case): > https://github.com/tristanls/alldata/blob/2580caa36c7ec11fe1b28a2e57485dd5dc7fec61/scripts/alldata-http-http-leveldb.js#L172 > > Whenever gossipmonger detects a new peer or a dead peer it notifies the > coordinator so that it isn't trying to replicate to dead nodes. Only > "transport" and "location" keys are ever gossiped because that's all the > information dissemination necessary for a coordinator (in this use case). > > On Sunday, October 20, 2013 11:03:58 AM UTC-5, Kevin Swiber wrote: >> >> Thanks, Tristan. This is helpful. I'll find time to give gossipmonger a >> test drive this week. Looks great. >> >> Sent from my iPhone >> >> On Oct 20, 2013, at 11:08 AM, Tristan Slominski <[email protected]> >> wrote: >> >> Since I've been asked to make the comparison a few times, it's become >> clearer what I think the difference between scuttlebutt and gossipmonger >> is. It actually starts to feel like comparing apples to oranges. >> >> scuttlebutt: "peer-to-peer replicatable data structure", two (maybe >> more?) scuttlebutts connect, negotiate the differences, and synchronize >> gossipmonger: endpoint that uses connection-less gossip protocol to find >> other gossipmongers, track whether they are live or not, and use >> "scuttlebutt reconciliation" strategy to disseminate information to other >> gossipmongers. >> >> Perhaps the above comparison is shorter and more useful :) >> >> There are other projects I'm working on that use gossipmonger as a >> component. When those become easier to consume, they might better >> illustrate the reason for gossipmonger's existence. >> >> Cheers! >> >> On Sunday, October 20, 2013 6:07:07 AM UTC-5, Floby wrote: >>> >>> Thanks for taking the time to write a detailed reply. It's much >>> appreciated. >>> >>> My point of view is that I'm not aware of all the details of the science >>> under the hood and I'm pretty happy this way for now. Dominic's >>> implementation provides a very small API surface which is in turn very >>> simple to learn (instanciate it, pipe it, use it). Extending his >>> implemntation is also much simpler and requires less knowledge of the >>> implementation details (only 2 methods to override). I haven't taken a >>> closer look at your module yet but it already seems to me that I have to >>> learn a whole lot of new concepts that I don't really need to know for a >>> basic use. >>> >>> My primary use case at the moment is synchronizing client side models >>> (in a browser) accross multiple users. I mostly had to combine scuttlebutt, >>> shoe and mux-demux. As you see, I'm only dealing with connected transports >>> (websockets, http) exposed as streams. The stream approach to scuttlebutt >>> makes it really easy to compose with other modules in my opinion. >>> >>> I'll make sure to give a try to your implementation for another project >>> of mine =) >>> >>> On Friday, 18 October 2013 16:06:20 UTC+2, Tristan Slominski wrote: >>>> >>>> Hey Kevin and Floby, >>>> >>>> Heh.. I have a hard time of thinking of a short answer to this, so >>>> please be patient. >>>> >>>> I wanted to write an open source gossipmonger for a while now. My >>>> initial implementation of gossip was closed source and at the time >>>> dominictarr's version didn't exist. That's not an argument for primacy or >>>> anything, I just want to highlight that I've been living with this >>>> implementation approach in my head for a long time, so my intuition has >>>> been indoctrinated in the approach demonstrated in gossipmonger. I was >>>> excited to see dominictarr's work (and substack's, as well as others in >>>> the >>>> community) as they shared their vision for peer-to-peer designs. So, when >>>> I >>>> needed an implementation for my current project, naturally I took a look >>>> at >>>> scuttlebutt, but overriding any reasons I can highlight below, I already >>>> had something that I knew worked, so I didn't really have too much >>>> incentive to learn something else that also worked :D. I'm as lazy as the >>>> next human. >>>> >>>> Having said that, I also don't know your use cases, so it's difficult >>>> to compare anything in the dark. All I can offer are some initial >>>> observations for what felt different in scuttlebutt compared to the >>>> gossipmonger implementation that I was familiar with for my particular use >>>> case. >>>> >>>> My use case/criteria for gossip are: >>>> >>>> 1. Promoting global awareness of peer-state. That is, given peer1, >>>> peer2, and peer3, only peer1 writes peer1 state, only peer2 writes peer2 >>>> state, and only peer3 writes peer3 state. However, I want peer2 to know >>>> the >>>> state of both peer1 and peer3, and so on. >>>> 2. Liveness determination (which is pretty much "phi accrual failure >>>> detection"). >>>> 3. Asynchronous communication. Specifically, I mean that I don't want >>>> to force a request-reply pattern of interaction. Even more specifically >>>> :), >>>> I don't want to force a connection-oriented pattern of interaction. You >>>> could also call this a "push-only" requirement. Part of the reason for >>>> this >>>> is that my initial gossip implementation was implemented over a messaging >>>> service (initial prototype was actually over pubnub ( >>>> http://www.pubnub.com/), although I moved away from that later, which >>>> ties into the next point). >>>> 4. Transport independence. That is, I wanted to be able to have an API >>>> that I could implement using messages in a bottle if I wanted to. >>>> >>>> So... >>>> >>>> "Scuttlebutt is always duplex. Scuttlebutt does a handshake..." ( >>>> https://github.com/dominictarr/scuttlebutt#gotchas). That doesn't meet >>>> my criteria #3. This also seems to me that it won't work over UDP. UDP >>>> implementation is hinted at in Introduction section of "Efficient >>>> Reconciliation and Flow Control for Anti-Entropy Protocols" ( >>>> http://www.cs.cornell.edu/home/rvr/papers/flowgossip.pdf): "Gossip >>>> protocols are designed to be non-invasive and have predictable >>>> performance, >>>> and for this a designer has to fix not only the gossip rate per >>>> participant >>>> but also the maximum size of gossip messages (e.g., maximum UDP packet >>>> size)." >>>> >>>> Another factor... >>>> >>>> "i.each(self.history(sources), function (data) {d._data(data)})" ( >>>> https://github.com/dominictarr/scuttlebutt/blob/8217ec7f96091838be3b56122d16176ba2b63fa6/index.js#L150). >>>> >>>> This indicates to me that the entire history since the last timestamp is >>>> sent over the wire (I may be wrong in this, but it is the impression that >>>> I >>>> got). Taking look at section 3.2, Scuttlebutt Reconciliation, of "Efficient >>>> Reconciliation and Flow Control >>>> for Anti-Entropy Protocols" ( >>>> http://www.cs.cornell.edu/home/rvr/papers/flowgossip.pdf) the authors >>>> would call this "precise reconciliation", which can result in sending >>>> updates that are unnecessary between the peers, "scuttlebutt >>>> reconciliation", on the other hand, converges more slowly, but "leaves >>>> room >>>> in the gossip message for deltas from other participants." Another >>>> consideration in that section is how to pick which deltas are sent in a >>>> gossip message. The authors outline "scuttle-breadth" and "scuttle-depth" >>>> approaches and say that "scuttle-depth" seemed to work better. I haven't >>>> been able to figure out where in the source code of scuttlebutt these >>>> things come into play (gossipmonger uses the "scuttle-depth" approach). To >>>> be fair, there may be subtleties in the implementation that I'm not >>>> seeing, >>>> but when in doubt I already had my implementation that met my criteria :). >>>> You could say I time-boxed my analysis of the source code. >>>> >>>> To summarize, for my four criteria, #2 was missing (or external), #3 >>>> seemed to not be satisfied, and #4 seemed to not be satisfied based on #3 >>>> not being satisfied. 3 out of 4 were missing, which was enough for me to >>>> open source an implementation I already had on hand. >>>> >>>> I hope this helps, and if my analysis was incorrect, then great, I'd >>>> love to have someone walk me through how my use case could be implemented >>>> in scuttlebutt. Scuttlebutt is great, and it had much more eyeballs for >>>> much longer on it. To be fair, I didn't ask dominictarr about any of this, >>>> but this goes again back to the fact that I already had an implementation, >>>> so it was easier/faster to release it. >>>> >>>> Cheers, >>>> >>>> Tristan >>>> >>>> On Friday, October 18, 2013 3:29:59 AM UTC-5, Floby wrote: >>>>> >>>>> Same as Kevin, >>>>> >>>>> I'm currently using Scuttlebutt and it's working great for my use >>>>> case. Can you point out the major difference or enhancements ? >>>>> >>>>> On Friday, 18 October 2013 02:22:07 UTC+2, Tristan Slominski wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> I'm happy to announce the initial release of Gossipmonger ( >>>>>> https://github.com/tristanls/gossipmonger). >>>>>> >>>>>> Gossipmonger is an implementation of the Scuttlebutt gossip protocol >>>>>> for real-time peer-to-peer state distribution (and liveness monitoring). >>>>>> Additionally, it has pluggable storage and transport mechanisms >>>>>> (currently >>>>>> it ships with in-memory storage and TCP transport), so it is open to be >>>>>> implemented in the browser as well as server-side. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> Tristan >>>>>> >>>>> -- >> -- >> Job Board: http://jobs.nodejs.org/ >> Posting guidelines: >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> You received this message because you are subscribed to the Google >> Groups "nodejs" group. >> To post to this group, send email to [email protected] >> To unsubscribe from this group, send email to >> [email protected] >> For more options, visit this group at >> http://groups.google.com/group/nodejs?hl=en?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "nodejs" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
