On Fri, Feb 3, 2017 at 1:15 PM, Edward Capriolo <[email protected]> wrote:
> > On Fri, Feb 3, 2017 at 1:13 PM, Dorian Ellerbe <[email protected]> > wrote: > >> Great ideas! >> >> 1) Regarding SWIM: Looks interesting, but I want to understand/research >> more. >> >> 2) +1 for HTTP. Anything to appeal to a larger audience of users and >> committers is good, especially if the dev overhead is relatively low. >> >> 5) Same idea as #2 above; though this will obviously require a larger >> effort. Maybe a functional implementation? >> >> 6) I've been reading a lot about Docker Swarm. We can spin up N instances >> of members and configure them in virtually (no pun) anyway we want. There >> are some excellent use cases for it and this is certainly one of them. >> >> On Mon, Jan 30, 2017 at 2:21 PM Edward Capriolo <[email protected]> >> wrote: >> >> > On Mon, Jan 30, 2017 at 1:03 PM, Sandeep More <[email protected]> >> > wrote: >> > >> > > For #3 the problem is a digest is not easy. The reason is that hosts >> > > communicate through each other. If there were three nodes A, B, and C, >> > and >> > > a message was sent from A -> B and then the message was sent from B >> -> C. >> > > Node B would be able to change the data and the digest. What I want >> to do >> > > is be able to ensure that the data is verified by A. This would be >> > > something like a PGP email. I want to verify that the message is >> > unaltered >> > > and that it is from a specific sender. >> > > >> > > SRM: I see, that would be complicated verifying the sender. I was >> > thinking >> > > of just verifying the signature on the hashes. Assuming we have a >> shared >> > > secret amongst all the nodes verifying a signature would not be too >> > > difficult. Verifying that the data comes from a specific node might be >> > > tricky. >> > > >> > > "It would be really cool if one could choose a custom data (like a >> > > String/Long value), I understand that this could be misused and >> > > misinterpreted as datastorage so may be there can be stricter limits >> on >> > the >> > > size of the custom payload. This might help Apps to integrate Gossip >> > > better." >> > > >> > > I am unclear about what you are saying here. We can already gossip >> > > arbitrary data between nodes. >> > > >> > > SRM: My bad, I missed this, thanks for pointing it out ! >> > > >> > > https://github.com/apache/incubator-gossip/blob/master/ >> > > src/test/java/org/apache/gossip/DataTest.java >> > > >> > > Thanks, >> > > Edward >> > > >> > > On Mon, Jan 30, 2017 at 12:45 PM, Edward Capriolo < >> [email protected] >> > > >> > > wrote: >> > > >> > > > On Mon, Jan 30, 2017 at 9:13 AM, Sandeep More < >> [email protected]> >> > > > wrote: >> > > > >> > > > > This is exciting ! >> > > > > >> > > > > #3, #2 and #1 especially look a great value add. >> > > > > >> > > > > On #3 I think signing the digest would be easier in short run, >> > > encrypting >> > > > > will mean involving complex keystore/truststore setup. >> > > > > >> > > > > May be you already covered this in Recipes or elsewhere but still >> > > putting >> > > > > it here: >> > > > > >> > > > > It would be really cool if one could choose a custom data (like a >> > > > > String/Long value), I understand that this could be misused and >> > > > > misinterpreted as datastorage so may be there can be stricter >> limits >> > on >> > > > the >> > > > > size of the custom payload. This might help Apps to integrate >> Gossip >> > > > > better. >> > > > > >> > > > > Just a thought ! >> > > > > >> > > > > Best, >> > > > > Sandeep >> > > > > >> > > > > On Sun, Jan 29, 2017 at 10:25 PM, Edward Capriolo < >> > > [email protected] >> > > > > >> > > > > wrote: >> > > > > >> > > > > > We currently have almost 10 open tickets for features / >> > improvements >> > > to >> > > > > > gossip, many are slated for the next release and we are on our >> way >> > to >> > > > be >> > > > > > ahead of schedule with that. >> > > > > > >> > > > > > I wanted to pick everyone's brain as to where we should aim for. >> > > > > > >> > > > > > I think some larger general directions are below: >> > > > > > >> > > > > > 1) SWIM. https://www.cs.cornell.edu/~as >> das/research/dsn02-swim.pdf >> > > > > > >> > > > > > This is a fairly large undertaking in terms of codifying the >> > protocol >> > > > and >> > > > > > keeping the project elegant with two Failure Detectors >> > > > > > >> > > > > > 2) HTTP as transport >> > > > > > >> > > > > > Having two transports in the codebase is simple. I think this >> will >> > go >> > > > > well >> > > > > > HTTP clients do all the session/persistence. Transplanting >> Gossip >> > to >> > > > > live >> > > > > > in a webapp wont be too bad but it might involve re-orging the >> > > project >> > > > a >> > > > > > bit into a multi-module maven project. I see a lot of upside >> here >> > > > > > >> > > > > > 3) Signed messages ( >> > https://issues.apache.org/jira/browse/GOSSIP-47) >> > > > > > >> > > > > > While I am not a super expert in keystores and such this >> strikes me >> > > as >> > > > > > interesting idea. Mostly because it allows peers to share info >> but >> > > also >> > > > > be >> > > > > > able to sign/verify/encrypt info as it moves between peers. I >> have >> > > > never >> > > > > > seen something quite like this so I think it is novel. >> > > > > > >> > > > > > 4) Recipes >> > > > > > >> > > > > > Building things like Ephemeral Nodes or Leader Election seem to >> be >> > a >> > > > good >> > > > > > fit. Gossip should not be a database so this is a hard line to >> > draw. >> > > > This >> > > > > > will take some research and expertise to implement correctly. >> > > > > > >> > > > > > 5) Second implementation >> > > > > > >> > > > > > Originally I planned to build out a second implementation in c, >> > node, >> > > > or >> > > > > > python. This seems less appealing to me at the moment, but if >> > Gossip >> > > > Java >> > > > > > gets too large/complex we will likely miss out window to do >> this. >> > > > > > >> > > > > > 6) something massive to spin up N instances in amazon and do >> > testing >> > > > > > >> > > > > > This is something that needs to happen, maybe in two parts. >> > > > > > >> > > > > > 7) Other ideas? Let them fly. >> > > > > > >> > > > > > Thanks, >> > > > > > Edward >> > > > > > >> > > > > >> > > > >> > > > For #3 the problem is a digest is not easy. The reason is that hosts >> > > > communicate through each other. If there were three nodes A, B, and >> C, >> > > and >> > > > a message was sent from A -> B and then the message was sent from B >> -> >> > C. >> > > > Node B would be able to change the data and the digest. What I want >> to >> > do >> > > > is be able to ensure that the data is verified by A. This would be >> > > > something like a PGP email. I want to verify that the message is >> > > unaltered >> > > > and that it is from a specific sender. >> > > > >> > > > "It would be really cool if one could choose a custom data (like a >> > > > String/Long value), I understand that this could be misused and >> > > > misinterpreted as datastorage so may be there can be stricter >> limits on >> > > the >> > > > size of the custom payload. This might help Apps to integrate Gossip >> > > > better." >> > > > >> > > > I am unclear about what you are saying here. We can already gossip >> > > > arbitrary data between nodes. >> > > > >> > > > https://github.com/apache/incubator-gossip/blob/master/ >> > > > src/test/java/org/apache/gossip/DataTest.java >> > > > >> > > > Thanks, >> > > > Edward >> > > > >> > > >> > >> > " Verifying that the data comes from a specific node might be tricky." >> > >> > For this feature to work the cluster nodes with either need one or more >> Pre >> > Shared Key or they would all have to agree on a (central) Certificate >> > Authority that could grant and verify. I would not expect this to be the >> > standard mode for most users, but I think it is fairly novel and would >> be >> > interesting to me. >> > >> > > For #6 there are likely a few apache projects products producing docker > images. I am open to implementation here. > Update: I realized when I setup the time based release I said 3 months, however I set the date as march 8th which was only two months in the future. LOL anyway great way to trick yourself. I set up the version 0.1.3 in Jira and moved many of these topics into that version.
