Is there any possibility of using Distributed CEP/Siddhi here? Because with Siddhi we can have some flexibility in the way we want to throttle and throttling is a common usecase of CEP. Its underline architecture also uses Hazelcast or Storm for distributed processing.
Regards Suho On Tue, Dec 24, 2013 at 8:54 AM, Manoj Fernando <[email protected]> wrote: > +1. Changing caller contexts in to a Hazlecast map would require some > significant changes to the throttle core, which may eventually be > re-written. > > Will update the design. > > Thanks, > Manoj > > > On Mon, Dec 23, 2013 at 4:09 PM, Srinath Perera <[email protected]> wrote: > >> Manoj, above plan look good. >> >> I chatted with Azeez, and we cannot register a Entry listener as I >> mentioned before because hazecast does not support entry listeners for >> atomic long. >> >> --Srinath >> >> >> On Mon, Dec 23, 2013 at 11:15 AM, Manoj Fernando <[email protected]> wrote: >> >>> Short update after the discussion with Azeez. >>> >>> - The need to re-write the throttle core is still at large, so the best >>> was to see how we can decouple the persistence logic from the throttle core >>> (at least as much as possible). >>> - A cluster updatable global counter will be included to the >>> ThrottleContext. The idea is that each node will periodically broadcast >>> the local counter info to the members in the cluster and the >>> ThrottleConfiguration will update the value of the Global counter summing >>> up the local counter values. >>> - The ThrottleConfiguration will also push the global counter values to >>> the Axis2 Configuration Context; a K, V pairs identified by the >>> ThrottleContext ID. >>> - A new platform component needs to be written to read the throttle >>> related Axis2 Config Context list and persist them periodically (duration >>> configurable). The throttle core will have no visibility into this >>> persistence logic, so this will be completely decoupled. >>> - So who should do the persistence? We can start with letting all nodes >>> to persist first, but later (or in parallel) we can improve the Hazlecast's >>> leader election (if that's not already there), so that the leader takes the >>> responsibility of persisting. >>> - The counters will be read off the persistence store at the time of >>> Hazlecast Leader election takes place? (An alternative is to load the >>> global counters at the init of ThrottleConfiguration but that means >>> coupling throttle core with persistence.) >>> >>> I will update the design accordingly. >>> >>> Any more thoughts or suggestions? >>> >>> Regards, >>> Manoj >>> >>> >>> On Thu, Dec 19, 2013 at 12:30 PM, Manoj Fernando <[email protected]>wrote: >>> >>>> +1. Let me setup a time. >>>> >>>> Regards, >>>> Manoj >>>> >>>> >>>> On Thursday, December 19, 2013, Srinath Perera wrote: >>>> >>>>> We need Azeez's feedback. Shall you, myself, and Azeez chat sometime >>>>> and decide on the first Arch design? >>>>> >>>>> >>>>> On Thu, Dec 19, 2013 at 11:55 AM, Manoj Fernando <[email protected]>wrote: >>>>> >>>>> Hi Srinath, >>>>> >>>>> That sounds like a much cleaner solution. We can perhaps use the >>>>> native map-store declarative [1] which I think does something similar. It >>>>> may sound a little silly to ask... but are we keeping Hazlecast active on >>>>> a >>>>> single node environment as well? :) Otherwise we will have to handle >>>>> persistence on a single node in a different way. This is with the >>>>> assumption of needing to persist throttle data on a single node >>>>> environment >>>>> as well (but questioning if we really need to do that is not totally >>>>> invalid IMO). >>>>> >>>>> Shall we go ahead with the Hazlecast option targeting cluster >>>>> deployments then? >>>>> >>>>> - Manoj >>>>> >>>>> [1] https://code.google.com/p/hazelcast/wiki/MapPersistence >>>>> >>>>> >>>>> On Thu, Dec 19, 2013 at 10:51 AM, Srinath Perera <[email protected]>wrote: >>>>> >>>>> One another way to do this use Hazelcast and then use "though cache" >>>>> or "Change listener's" in Hazecast for persistence. >>>>> >>>>> --Srinath >>>>> >>>>> >>>>> On Tue, Dec 17, 2013 at 4:49 PM, Manoj Fernando <[email protected]>wrote: >>>>> >>>>> +1 for persisting through a single (elected?) node, and let Hazlecast >>>>> do the replication. >>>>> >>>>> I took into consideration the need to persist periodically instead of >>>>> at each and every request (by spawning a separate thread that has access >>>>> to >>>>> the callerContext map)... so yes... we should think in the same way for >>>>> replicating the counters across the cluster as well. >>>>> >>>>> Instead of using a global counter, can we perhaps use the last updated >>>>> timestamp of each CallerContext? It's actually not a single counter we >>>>> need to deal with, and each CallerContext instance will have separate >>>>> counters mapped to their throttling policy AFAIK. Therefore, I think its >>>>> probably better to update CallerContext instances based on the last update >>>>> timestamp. >>>>> >>>>> WDYT? >>>>> >>>>> If agree, then I need to figure out how to make delayed replication on >>>>> hazlecast (is it through the hazelcast.heartbeat.interval.seconds config >>>>> item?) >>>>> >>>>> Regards, >>>>> Manoj >>>>> >>>>> >>>>> On Tue, Dec 17, 2013 at 4:22 PM, Srinath Perera <[email protected]>wrote: >>>>> >>>>> We need to think it a cluster setup do we need persistence as well? As >>>>> we can have replication using Hazelcast? >>>>> >>>>> If we need persistence, I think it is a good if a single node persists >>>>> the current throttling values, and if that node fails, someone else takes >>>>> it place? >>>>> >>>>> Current implementation sync the values across the cluster per each >>>>> message, which introduce significant overhead. I think we should go to a >>>>> model where each node collects and update the values once few seconds. >>>>> >>>>> idea is >>>>> 1) there is a global counter, that we use to throttle >>>>> 2) Each node keep a global counter, and periodically it update the >>>>> global counter using value in location counter and reset the counter and >>>>> read the current global counter. >>>>> 3) Until next update, each node make decisions based on local global >>>>> counter values it has read already >>>>> >>>>> This will mean that the throttling will throttle close to the limit, >>>>> not exactly at the limit. However, IMHO, that is not a problem for >>>>> throttling usecase. >>>>> >>>>> --Srinath >>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Dec 16, 2013 at 7:20 PM, Manoj Fernando <[email protected]> wrot >>>>> >>>>> >>>> >>>> -- >>>> Manoj Fernando >>>> Director - Solutions Architecture >>>> >>>> Contact: >>>> LK - +94 112 145345 >>>> Mob: +94 773 759340 >>>> www.wso2.com >>>> >>>> >>> >>> >>> -- >>> Manoj Fernando >>> Director - Solutions Architecture >>> >>> Contact: >>> LK - +94 112 145345 >>> Mob: +94 773 759340 >>> www.wso2.com >>> >> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> http://people.apache.org/~hemapani/ >> http://srinathsview.blogspot.com/ >> > > > > -- > Manoj Fernando > Director - Solutions Architecture > > Contact: > LK - +94 112 145345 > Mob: +94 773 759340 > www.wso2.com > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *S. Suhothayan* Associate Technical Lead, *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
