Re: Contribution to apache gossip

[Edward Capriolo]

You may have noticed that I broke out the task into an umbrella task. I
submitted a PR with one possibly implementation of implementing a "geo"
node chooser and created some tickets to add others. I also hinted at some
ways of doing the other components.

On Thu, Aug 10, 2017 at 10:48 PM, Edward Capriolo 
wrote:

>
> On Thu, Aug 10, 2017 at 4:27 AM, Ankur Agrawal 
> wrote:
>
>> Hi Edward
>>
>> Thanks for sharing your notes.
>>
>> Most of what you have said makes sense to me as far as functionality is
>> concerned. I need to take a stab at understanding the code flow again, and
>> think through the entire flow, which might take a day or two. After that I
>> will come up with a more detailed design, and maybe more questions :)
>>
>> Thanks
>> Ankur
>>
>> On Wed, Aug 9, 2017 at 6:17 PM, Edward Capriolo 
>> wrote:
>>
>>>
>>>
>>> On Wed, Aug 9, 2017 at 6:32 PM, Ankur Agrawal >> > wrote:
>>>
 Hi Edward,

 Thanks for your response. I spent sometime understanding about the
 Gossip protocol, and the apache-gossip codebase.

 About Gossip-70, I am trying to understand the requirement.

 Does this look like a good blueprint to you:

- Each node is initialized with a n-dimensional co-ordinate.
- When a node creates its membership list, it gets the co-ordinates
of its members.
- Each node can draw a logical graph based on the co-ordinates of
its members.
- This topology can be used by the application that is using
gossip, to forward its updates/messages and to decide quorum based on a
formula.

 Do you think my understanding of basic requirements is correct?

 Thanks
 Ankur

 On Mon, Jul 31, 2017 at 7:58 AM, Edward Capriolo  wrote:

> On Sun, Jul 30, 2017 at 10:59 PM, Ankur Agrawal <
> ankuragrawal...@gmail.com>
> wrote:
>
> > Hi
> >
> > Not sure if my previous email one week ago did actually get
> delivered. So I
> > apologize for resenting in advance.
> >
> > My name is Ankur. I am a student, and I am keenly interested in, and
> want
> > to contribute to this project for implementation of Gossip protocol.
> >
> > I have some job experience in designing instance migration services
> for a
> > popular public cloud infrastructure, and distributed data backup and
> > recovery services. I think this project will help me learn a lot,
> and I can
> > also contribute while learning.
> >
> > I saw that GOSSIP-70 (https://issues.apache.org/jir
> a/browse/GOSSIP-70) is
> > still open. It was mentioned as a project in GSoC 2017 too. If this
> is
> > indeed open, and you guys think I can start with this, it will be
> great.
> > Please note that I am not participating in GSoC and I want to
> contribute on
> > a individual capacity.
> >
> > If there are other open issues where you are looking for
> contribution too,
> > do let me know, and I can get started with that.
> >
> > Also, I am wondering if there is any interest in implementation of
> Gossip
> > in other languages like python or C++.
> >
> > I look forward to hear from you guys.
> >
> > Thanks
> > Ankur
> >
>
> Thanks for your interest!
>
> "Also, I am wondering if there is any interest in implementation of
> Gossip
> in other languages like python or C++."
>
> Originally we planned to port Gossip to multiple languages but there
> has
> not been much interest in that. Currently the code base is evolving and
> porting would be challenging. I think that is a lot for us to bite off
> at
> this point.
>
> GOSSIP-70 is a fine place to start. We have not fleshed that ticket out
> enough. Starting a conversation in the ticket or on the mailing list
> would
> be a good thing to do before we start coding. I like the idea of using
> a
> formula to pick a quorum of nodes and doing an operation across them.
> This
> can be easily extracted but we need to figure out what the first step
> is
> and what a final target might be.
>
> Thanks,
> Edward
>

>>> Your understanding is mostly in line with my thinking.  I will started
>>> some high level design but do not have the interfaces at a point such that I
>>> was ready for a commit. I will share my notes.
>>>
>>> >> Each node is initialized with a n-dimensional co-ordinate.
>>>
>>> That is one route but we can go more simple then that. We need an
>>> interface:
>>>
>>> interface chooser {
>>>public List generateEndpoints(string key, List
>>> live, List dead, LocalNode me);
>>> }
>>>
>>> The most simple implementation is we can hash the node's id into a line,
>>> or circular queue. We can 

[jira] [Commented] (GOSSIP-38) Multiple async GossipListeners

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/GOSSIP-38?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133136#comment-16133136
 ] 

ASF GitHub Bot commented on GOSSIP-38:
--

Github user edwardcapriolo commented on a diff in the pull request:

https://github.com/apache/incubator-gossip/pull/66#discussion_r133989277
  
--- Diff: 
gossip-base/src/main/java/org/apache/gossip/manager/GossipMemberStateRefresher.java
 ---
@@ -112,10 +115,16 @@ public boolean 
processOptimisticShutdown(Entry l) {
 if (s.getShutdownAtNanos() > l.getKey().getHeartbeat()) {
   members.put(l.getKey(), GossipState.DOWN);
   if (l.getValue() == GossipState.UP) {
-listener.gossipEvent(l.getKey(), GossipState.DOWN);
+for (GossipListener listener: listeners)
--- End diff --

We have a number of executors around the code. You should follow the 
pattern used elsewhere to setup the pool, add metrics, deal with rejecting, etc


> Multiple async GossipListeners
> --
>
> Key: GOSSIP-38
> URL: https://issues.apache.org/jira/browse/GOSSIP-38
> Project: Gossip
>  Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Pooya Salehi
> Fix For: 0.1.3
>
>
> Currently Gossip allows us to specific 0-1 synchronous event listeners. 
> In Hama:
> https://github.com/apache/hama/blob/master/core/src/main/java/org/apache/hama/monitor/fd/NodeEventListener.java
> They can attach multiple async listeners. I like this design better unsure if 
> we should support both or only the async kind.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)