Hi Edward, I prepared a draft design document for the vote based lock implementation. Could you please give me a feedback regarding the design?
https://docs.google.com/document/d/1sKbDs3aLcJxjKeEjXVRt9BnI1zKIPQTDN6RkDL_jEZc/edit?usp=sharing <https://docs.google.com/document/d/1sKbDs3aLcJxjKeEjXVRt9BnI1zKIPQTDN6RkDL_jEZc/edit?usp=sharing> Thanks *Mirage Abeysekara* Undergraduate Computer Science and Engineering University of Moratuwa Twitter: https://twitter.com/MiRAGECreator GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara On 23 March 2017 at 07:53, Edward Capriolo <[email protected]> wrote: > On Wed, Mar 22, 2017 at 3:33 PM, Mirage Abeysekara < > [email protected]> > wrote: > > > Hello Edward, > > > > Why do we need a distributed locking mechanism, if the gossip provides > > eventual consistency model? > > Can you please give me a bit of explanation or simple use-case? > > > > Thanks > > > > > > *Mirage Abeysekara* > > Undergraduate > > Computer Science and Engineering > > University of Moratuwa > > Twitter: https://twitter.com/MiRAGECreator > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > On 18 March 2017 at 21:50, Edward Capriolo <[email protected]> > wrote: > > > > > On Thu, Mar 16, 2017 at 5:16 AM, Mirage Abeysekara < > > > [email protected]> > > > wrote: > > > > > > > Hi Edward, > > > > > > > > I updated the jira ticket with design documentation link. > > > > The time period for GSoC is three months. I would like to be work > with > > > > another ticket. Can you assign me for a suitable ticket? > > > > > > > > Thanks > > > > > > > > *Mirage Abeysekara* > > > > Undergraduate > > > > Computer Science and Engineering > > > > University of Moratuwa > > > > Twitter: https://twitter.com/MiRAGECreator > > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > > > On 13 March 2017 at 18:47, Edward Capriolo <[email protected]> > > > wrote: > > > > > > > > > I assigned the ticket to you. I labeled it GSOC 2017. Please > > link/copy > > > > your > > > > > design docs into the ticket. It would be good to have another > ticket > > > > > earmarked to you. I do not know how long GSOC is but with the speed > > you > > > > did > > > > > the last ticket I do not anticipate pub/sub taking a month. > > > > > > > > > > On Mon, Mar 13, 2017 at 6:16 AM, Mirage Abeysekara < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hello Edward, > > > > > > > > > > > > I updated the document with changes. Here is the expected design > of > > > the > > > > > > classes. > > > > > > https://drive.google.com/file/d/0B8l-mZ- > > > pbfPmNWh2cU92NVhEMEk/view?usp= > > > > > > sharing > > > > > > > > > > > > <https://drive.google.com/file/d/0B8l-mZ- > > > > pbfPmNWh2cU92NVhEMEk/view?usp= > > > > > > sharing> > > > > > > > > > > > > Thanks. > > > > > > > > > > > > *Mirage Abeysekara* > > > > > > Undergraduate > > > > > > Computer Science and Engineering > > > > > > University of Moratuwa > > > > > > Twitter: https://twitter.com/MiRAGECreator > > > > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > > > > > > > On 12 March 2017 at 00:56, Edward Capriolo < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > I added some suggestions/ changes to your document. Looks good > on > > > my > > > > > end. > > > > > > > Does anyone else have any suggestions? > > > > > > > > > > > > > > On Fri, Mar 10, 2017 at 2:31 AM, Mirage Abeysekara < > > > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > I prepared a draft design document[1] for the above jira > ticket > > > > based > > > > > > on > > > > > > > my > > > > > > > > understanding of the project. I would appreciate if you could > > > give > > > > me > > > > > > > some > > > > > > > > suggestions or feedback for the design. > > > > > > > > > > > > > > > > [1] https://docs.google.com/document/d/ > > > > 1z1jZIaNqot8SXrelajd1aCBT1q_ > > > > > > > > KVHrw5DV2RGJfJtQ/edit > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > *Mirage Abeysekara* > > > > > > > > Undergraduate > > > > > > > > Computer Science and Engineering > > > > > > > > University of Moratuwa > > > > > > > > Twitter: https://twitter.com/MiRAGECreator > > > > > > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Mirage, > > > > > > Sorry I just saw this. I think the important thing is the proposal > should > > > be interersting to you. Thus I do not want to "assign" things. > > > > > > I would say the best idea is to look into more data structures: > > > http://ieeexplore.ieee.org/document/7328521/ > > > > > > Or possibly developing some form of lock management: > > > http://www.exploredatabase.com/2014/04/distributed-lock- > > > manager-concurrency.html > > > > > > Or building higher level datastructures ontop of CRDTs > > > https://ignite.apache.org/features/datastructures.html > > > > > > To do these we need to solve/handle a couple of issues. > > > > > > Building something that does a leader election makes sense as well. > > > > > > the open question in my mind is will we implement something like this: > > > > > > https://issues.apache.org/jira/browse/VXQUERY-180?jql= > > > labels%20%3D%20gsoc2017 > > > > > > > > > Where machine do direct communication vs lazy gossip replication. > > > > > > Also I have been thinking hard about the per-node vs shared hash maps. > I > > > thing what we need to there is unify those concepts and make the key of > > the > > > hashmap be useful for expressing more arbitrary concepts like "only > > > replicate this to nodes name xyz" or "only replicate this to nodes in > > > datacenter b" or "dont replicate this at all it is only local data" > > > > > > Having a distributed structure that I can lock is most interesting to > me. > > > Even if the lock takes 30 seconds to acquire by reaching a quorum of > > nodes, > > > wondering if we can implement something like this: > > > > > > https://www.google.com/webhp?sourceid=chrome-instant&ion=1&; > > > espv=2&ie=UTF-8#q=bakery+algorithm&* > > > > > > Anyway there is a lot of ideas there maybe you can take one in a more > > > specific direction. > > > > > > > https://issues.apache.org/jira/browse/GOSSIP-75 > > Ok lets take a simple example. Imagine I have some datum like this: > > { "name":"computePI", "workers": "4" } > > The question becomes how do we now divide up this work? How can we make > sure 4 and only 4 nodes take on this work. > > One possibility is voting. > > Node2 starts the process > createSharedData( name:computePi/slot1, value: "node2svote:node2" ) > now as this passes message passed to node3, node3 will append to the crdt > > createSharedData( > name:computePi/slot1, value: "node2svote:node2" , > name:computePi/slot1, value: "node3svote:node2" > ) > > Now, anyone who is interested in this lock/ can keep polling until number > of votes > half cluster. But there is other ways to do this. Possibly we > use a technique like that to elect a leader who is a decision maker for N > seconds, then we elect again. Or maybe this is where that geometric quorum > ticket comes in, maybe we do not attempt to gossip this, maybe we pick N > nodes and the communicate directly. >
