One minor correction - even serial AsyncEventListener's are only fired on a single node for each event. Serial AsyncEventListener's have their own concept of a primary, and the primary dispatches all events until it fails.
In this case I do think the parallel AsyncEventListener is probably the better option so you can scale out. -Dan On Thu, Jan 19, 2017 at 11:13 AM, Paul Perez <[email protected]> wrote: > Hi Url > That is a very good idea > I will test it and let you know. > Thank you everyone I really appreciate your help and your comments. > > Sent from my mobile phone > Paul Perez > Pymma > On Jan 19, 2017, at 19:00, Udo Kohlmeyer <[email protected]> wrote: >> >> Hi there Paul. >> >> We will assume that you are using partitioned regions. In partitioned >> regions you have the notion of "primary" and "redundant" data copies. Any >> CUD (create,update,destroy) operations will ALWAYS only happen on the >> primary node. Which means that with an AsyncEventListener, it will only >> ever "fire" on the primary data node. >> >> So no, you will not have the AsyncEventListener fire 3 times. >> >> With a replicate region, the AsyncEventListener will fire 3 times. >> >> The concept of serial vs parallel just means the amount to >> threads/executors that each AsyncEventListener will use. With serial, there >> will only be 1. With parallel you could have many threads, but once again >> an event will only ever be processed by one of the AsyncEventListener >> threads. (if you are using partitioned regions). >> >> You can try this out if you want. >> >> --Udo >> >> On 1/19/17 10:42, Paul Perez wrote: >> >> Hello Michael >> >> >> >> I did not see your answer before replying to Udo so may be in my last >> email I made mistakes and wrote wrong things. >> >> We also though about AsyncEventListeners but we found a difficulty in. >> >> Geode documentation says: >> >> *“You can configure an AsyncEventQueue to be either serial or parallel. A >> serial queue is deployed to one Geode member, and it delivers all of a >> region’s events, in order of occurrence, to a configured AsyncEventListener >> implementation. A parallel queue is deployed to multiple Geode members, and >> each instance of the queue delivers region events, possibly simultaneously, >> to a local AsyncEventListener implementation.” * >> >> >> >> Let’s say that for scalability reason we have 3 members in our >> aggregation system, and we implement the aggregation process in the >> Listener. We understand that it will be invoked 3 times. And since an >> aggregation is not a stateless process the aggregation will be wrong. >> >> >> >> May be I’m wrong and I did not understand the documentation. I would be >> very happy with that. >> >> >> >> Please let me know >> >> >> >> >> >> Best regards >> >> >> >> Paul >> >> >> >> *From:* Michael Stolz [mailto:[email protected] <[email protected]>] >> *Sent:* 19 January 2017 16:55 >> *To:* [email protected] >> *Cc:* [email protected]; [email protected] >> *Subject:* Re: Send an asynchronous event to one client among many >> >> >> >> Instead of hopping out to a client, you could get horizontal scale and >> asynchronous processing by using an AsyncEventListener in the servers. That >> will take care of multi-threading and queuing and all the plumbing, and you >> just go ahead and write your processing code and deploy it as >> AsyncEventListeners. This gives you guaranteed ordering semantics for each >> key as well. >> >> >> >> I *think* it even gives you a notion of H/A so that if the primary fails >> the queued messages will be processed by a secondary. (I know the WAN >> Gateway does and it uses pretty much the same plumbing under the covers). >> >> >> >> >> -- >> >> Mike Stolz >> >> Principal Engineer, GemFire Product Manager >> >> Mobile: 631-835-4771 <(631)%20835-4771> >> >> >> >> On Thu, Jan 19, 2017 at 11:21 AM, Udo Kohlmeyer <[email protected]> wrote: >> >> Hi there Paul, >> >> Firstly, your use case looks really interesting and hope to see a few >> more posts on how you use Geode further. Keep us informed we like to hear >> what you guys are doing with GEODE! :) >> >> The subscription or CQ (continuous query) paradigm is, as stated, a 1:1 >> relationship. When a client registers interest on a region that client will >> be notified. This is more of a topic semantic rather than a queue semantic. >> >> Although this is not the first time I've heard the request for this kind >> of functionality. To best explain why GEODE, currently, implements the 1:1 >> relationship has got to do with guaranteed delivery and in-order delivery. >> If we use a queue semantic, with multiple clients being able to process >> data in a balanced manner, we end up with potential out-of-order processing >> of messages. In addition to that it now becomes significantly harder to >> track and n deal with client failures and the potential replaying of >> messages. >> >> But that said, I have seen other users resolve this problem and could >> detail some approaches in a later correspondence if you'd like. >> >> --Udo >> >> >> >> >> >> >> >> On 1/19/17 03:39, Paul Perez wrote: >> >> Hello All, >> >> >> >> As explained in a previous email, we try to use Geode to process and >> aggregate a stream of Traces. Our requirement is to process billions of >> simple traces every day. >> >> We imagine the aggregation process in many steps. >> >> One: traces are generated by a tiers tools and stored in a first geode >> region >> >> Two: once a trace put in the first region we use the async event >> feature to invoke a client that executes the first aggregation steps. Then >> the result will be put in a second region. >> >> Three: the second aggregation step is in the same way, when traces are >> put in the second region, then an asynchronous event is sent to the client >> to execute the second part of the aggregation etc.… >> >> >> >> For scalability purposes, we plan to use many clients that could receive >> the events and execute the aggregation and put the results back to Geode. >> >> Consequently, as far as we understand the documentation, when an entry is >> put in a region, each client that registered an interest receives an event >> and aggregate the trace. So, the trace will be aggregated many times. >> >> >> >> My question is: If many clients are registered, could we configure the >> region to send randomly, the event to one client only. >> >> A subsidiary question: Do we have the same behaviour with the function >> execution feature or it could be an alternative in that case >> >> Thank you for your help >> >> Best regards >> >> >> >> Paul >> >> >> >> >> >> >> >> >>
