On Wed, Apr 12, 2017 at 2:06 PM, Gary Dusbabek <[email protected]> wrote:
> I have a few questions: > > 1. Interchangeable Transports: Is there support for adding transports > beyond UDP? I can see some utility for TCP-based transports. > > 2. Interchangeable Protocols: JSON is great for some things, but IME > nothing beats a good binary protocol. Also, would protocol versioning be a > good idea. Consider the scenario of many thousands of nodes being upgraded > in rolling fashion. > > 3. Layered Approach: This leads me to the final topic, which is to propose > adopting an architecture that is more layered. Doing so will lead to code > that is inherently more testable and usable as a library in other projects. > I am mainly thinking in the context of OSI layers 5-7 but I'm open to > suggestions. > > If so, I'll work up a design, create JIRAs and start the work. > > Thx, > > Gary > Gary. Good to hear from you. Good timing. 1) I just started working on https://issues.apache.org/jira/browse/GOSSIP-50. looking at the change briefly I noticed a few things. I am looking at spring boot. PassiveGossipThread needs to be modified slightly so that it can run as an http servlet. This is not a huge issue. PassiveGossipThread is essentially blocking on standard in and calling GossipCore.recieve() The other side of this is GossipCore.send() needs to pick a client based on the URI of the partner. IE if talking to http use a http client. 2) The code had a piece that constructed a packet with a 4 byte header and payload, I took that out because I found it was less byte copies to have Jackson read the object from the wire without having to read a byte[] and strip 4 characters, then decode the payload from the byte array. I am ok with adding the header (or some other header) back. The challenge I see is that our payload could contain any java object so the binary protocol has to be able to deconstruct/reconstruct types. This takes some mucking around with json modules at times to get what you want, and if you had one user supporting json and another kyro this would be a lot of work. But to wrap up yes we need something better then current. 3) In general I do not have any problem with this. One thing I found with GossipCore: GossipCore just became a place for the network stuff, and also the per node and shared hash maps landed in there. I felt the abstraction was wrong, from gossipmanager it only meant chained method calls. It made funky OOD because features cross cut layers. I recently cut some of this out GossipService->GossipManager->GossipCore. Now it is just GossipManager->GossipCore. I probably want to rename GossipCore to GossipComm and move those data maps back into GossipManager. In terms of OSI layers I can only think of a few concrete examples, IE if I was gossiping over multicast pieces would be different. Again I am open to any refactors needed as features go forward. Even some not attached to a feature that makes things cleaner. For #1 maybe we can address that as part of gossip-50, for #2 you can tackle that now. For #3 that looks like a case by case.
