We need Azeez's feedback. Shall you, myself, and Azeez chat sometime and decide on the first Arch design?
On Thu, Dec 19, 2013 at 11:55 AM, Manoj Fernando <[email protected]> wrote: > Hi Srinath, > > That sounds like a much cleaner solution. We can perhaps use the native > map-store declarative [1] which I think does something similar. It may > sound a little silly to ask... but are we keeping Hazlecast active on a > single node environment as well? :) Otherwise we will have to handle > persistence on a single node in a different way. This is with the > assumption of needing to persist throttle data on a single node environment > as well (but questioning if we really need to do that is not totally > invalid IMO). > > Shall we go ahead with the Hazlecast option targeting cluster deployments > then? > > - Manoj > > [1] https://code.google.com/p/hazelcast/wiki/MapPersistence > > > On Thu, Dec 19, 2013 at 10:51 AM, Srinath Perera <[email protected]> wrote: > >> One another way to do this use Hazelcast and then use "though cache" or >> "Change listener's" in Hazecast for persistence. >> >> --Srinath >> >> >> On Tue, Dec 17, 2013 at 4:49 PM, Manoj Fernando <[email protected]> wrote: >> >>> +1 for persisting through a single (elected?) node, and let Hazlecast do >>> the replication. >>> >>> I took into consideration the need to persist periodically instead of at >>> each and every request (by spawning a separate thread that has access to >>> the callerContext map)... so yes... we should think in the same way for >>> replicating the counters across the cluster as well. >>> >>> Instead of using a global counter, can we perhaps use the last updated >>> timestamp of each CallerContext? It's actually not a single counter we >>> need to deal with, and each CallerContext instance will have separate >>> counters mapped to their throttling policy AFAIK. Therefore, I think its >>> probably better to update CallerContext instances based on the last update >>> timestamp. >>> >>> WDYT? >>> >>> If agree, then I need to figure out how to make delayed replication on >>> hazlecast (is it through the hazelcast.heartbeat.interval.seconds config >>> item?) >>> >>> Regards, >>> Manoj >>> >>> >>> On Tue, Dec 17, 2013 at 4:22 PM, Srinath Perera <[email protected]>wrote: >>> >>>> We need to think it a cluster setup do we need persistence as well? As >>>> we can have replication using Hazelcast? >>>> >>>> If we need persistence, I think it is a good if a single node persists >>>> the current throttling values, and if that node fails, someone else takes >>>> it place? >>>> >>>> Current implementation sync the values across the cluster per each >>>> message, which introduce significant overhead. I think we should go to a >>>> model where each node collects and update the values once few seconds. >>>> >>>> idea is >>>> 1) there is a global counter, that we use to throttle >>>> 2) Each node keep a global counter, and periodically it update the >>>> global counter using value in location counter and reset the counter and >>>> read the current global counter. >>>> 3) Until next update, each node make decisions based on local global >>>> counter values it has read already >>>> >>>> This will mean that the throttling will throttle close to the limit, >>>> not exactly at the limit. However, IMHO, that is not a problem for >>>> throttling usecase. >>>> >>>> --Srinath >>>> >>>> >>>> >>>> >>>> On Mon, Dec 16, 2013 at 7:20 PM, Manoj Fernando <[email protected]>wrote: >>>> >>>>> Attaching Gdoc as a pdf. >>>>> >>>>> - Manoj >>>>> >>>>> >>>>> On Mon, Dec 16, 2013 at 9:15 AM, Manoj Fernando <[email protected]>wrote: >>>>> >>>>>> All, >>>>>> >>>>>> We have a requirement for $subject. Like to hear your thoughts first >>>>>> on the following plan, and setup a review session accordingly. >>>>>> >>>>>> Google doc @ [1] with permissions to comment. >>>>>> >>>>>> *Background* >>>>>> Throttling is a core carbon component that provides API throttling >>>>>> across the platform. The current implementation supports Role and >>>>>> Concurrency based throttling which is used by products for more business >>>>>> specific use cases. For example, the APIM uses the throttling framework >>>>>> to >>>>>> provide throttling support at 3 levels. >>>>>> >>>>>> - Application Level - Policy is applied to the whole Application >>>>>> (overrides any policy violations at the other 2 levels) >>>>>> - API Level - Policy is applied at each API level (overrides any >>>>>> policy violations at API Resource level) >>>>>> - API Resource Level - Policy is applied at each API resource >>>>>> (i.e. GET, POST, etc.) >>>>>> >>>>>> >>>>>> *Problem* >>>>>> At present, the core carbon framework does not persist the runtime >>>>>> throttling data. For example, a role based APIM throttling policy may >>>>>> specify that 50 requests be handled per minute, and if the APIM gateway >>>>>> crashes at the 50th second having served 40 requests, a restart will >>>>>> cause >>>>>> in APIM providing the full quota once the node is restarted. >>>>>> >>>>>> >>>>>> *Current Design* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> - ThrottleContext is initialized by APIThrottleHandler (in the >>>>>> case of API Manager) at the time of the first authenticated request >>>>>> hitting >>>>>> the gateway. >>>>>> - The APIThrottleHandler uses the ThrottleFactory (carbon core >>>>>> class) to instantiate a ThrottleContext object. >>>>>> - ThrottleContext keeps a map of CallerContext objects of which >>>>>> the runtime throttle counters are kept, corresponding to each policy >>>>>> definition (e.g. A throttle scenario mapping the tier policy ‘Gold’ >>>>>> will >>>>>> initiate a CallerContext at the first instance of the policy is >>>>>> matched.) >>>>>> - For every new CallerContext instance, the ThrottleContext will >>>>>> push that CallerContext instance to a Map. >>>>>> - ThrottleContext exposes the ‘addCallerContext’ and >>>>>> ‘removeCallerContext’ methods to add and to cleanup the expired >>>>>> context >>>>>> objects. >>>>>> - CallerContext keeps the caller count, and access times related >>>>>> to the Caller. >>>>>> - In the case of API Manager, each caller instance (based on the >>>>>> tier configuration), access the ThrottleContext using the >>>>>> doRoleBasedAccessThrottling and doThrottleByConcurrency methods. >>>>>> >>>>>> >>>>>> *Implementing Persistence* >>>>>> >>>>>> >>>>>> - ThrottleContext is independently initialized by any component >>>>>> using the throttling framework. >>>>>> - What needs to be persisted is the CallerContext map together >>>>>> with the initiator attributes (i.e. TrottleID) >>>>>> - An option is to spawn a separate Thread on the ThrottleContext >>>>>> constructor that will have access to the CallerContext map. >>>>>> - A new Persistence DAO (i.e. ThrottleContextPersister class), >>>>>> can access the cached CallerContext instances using the >>>>>> ThrottleUtil.getThrottleCache(). >>>>>> - This ThrottleContextPersister needs to clean up the old caller >>>>>> contexts entries on the DB before persisting the new caller entries. >>>>>> - Persistence interval can be made configurable (carbon.xml ?). >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *Q&A* >>>>>> >>>>>> 1. How does this work on a clustered environment? >>>>>> Irrespective of the node running on a cluster or not, we need to >>>>>> persist the CallerContext map. >>>>>> >>>>>> Option A : Persist the caller context on an elected node in the >>>>>> cluster given the fact that we can use Hazelcast to distribute the >>>>>> callercontext map across the cluster nodes. >>>>>> >>>>>> Option B : Each node to independently persist their caller maps >>>>>> against the node info. In this way, we will not have to rely on cluster >>>>>> replication of the caller context map. >>>>>> >>>>>> 2. How does DB persistence done at the carbon core level? >>>>>> [TODO : Find out how persistence is handled at the carbon core level.] >>>>>> >>>>>> 3. Are there any product specific objects that need to be persisted >>>>>> as well? >>>>>> AFAIK, no we do not need to. If you take the APIM for example, the >>>>>> tier config gets loaded at the server startup and using the tier IDs we >>>>>> should be able to initialize (load) the CallerContext map corresponding >>>>>> to >>>>>> that scenario. >>>>>> >>>>>> 4. How often the CallerContext map need to be persisted? >>>>>> As a thought, we should persist the CallerContext every 5-10 seconds >>>>>> (IMO this should be a medium prio thread). Can we make this value >>>>>> configurable? >>>>>> >>>>>> 5. Any chance of losing most recent runtime throttle info as we are >>>>>> not persisting on each request? >>>>>> Yes there is. But this is a trade-off between performance and the >>>>>> requirement to persist throttle conters. Making the throttle persistence >>>>>> interval configurable is a measure to control this. >>>>>> >>>>>> >>>>>> 6. What needs to be persisted? >>>>>> The following at a minimum >>>>>> >>>>>> ID : string /* The Id of caller */ >>>>>> nextAccessTime : long /* next access time - the end of >>>>>> prohibition */ >>>>>> firstAccessTime : long /* when caller came across the on first time >>>>>> */ >>>>>> nextTimeWindow : long /* beginning of next unit time period */ >>>>>> count : int /* number of requests */ >>>>>> >>>>>> If we opt to use Option B for handling throttle persistence in a >>>>>> cluster we will have to persist the nodeID in addition to these. >>>>>> >>>>>> >>>>>> >>>>>> [1] >>>>>> https://docs.google.com/a/wso2.com/document/d/1AQOH-23jM37vjtzqoWg7vokUTsyaWh3eJLLoYQXYlf0 >>>>>> >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Regards, >>>>>> Manoj >>>>>> -- >>>>>> Manoj Fernando >>>>>> Director - Solutions Architecture >>>>>> >>>>>> Contact: >>>>>> LK - +94 112 145345 >>>>>> Mob: +94 773 759340 >>>>>> www.wso2.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Manoj Fernando >>>>> Director - Solutions Architecture >>>>> >>>>> Contact: >>>>> LK - +94 112 145345 >>>>> Mob: +94 773 759340 >>>>> www.wso2.com >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> http://people.apache.org/~hemapani/ >>>> http://srinathsview.blogspot.com/ >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Manoj Fernando >>> Director - Solutions Architecture >>> >>> Contact: >>> LK - +94 112 145345 >>> Mob: +94 773 759340 >>> www.wso2.com >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> ============================ >> Srinath Perera, Ph.D. >> Director, Research, WSO2 Inc. >> Visiting Faculty, University of Moratuwa >> Member, Apache Software Foundation >> Research Scientist, Lanka Software Foundation >> Blog: http://srinathsview.blogspot.com/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Manoj Fernando > Director - Solutions Architecture > > Contact: > LK - +94 112 145345 > Mob: +94 773 759340 > www.wso2.com > -- ============================ Srinath Perera, Ph.D. Director, Research, WSO2 Inc. Visiting Faculty, University of Moratuwa Member, Apache Software Foundation Research Scientist, Lanka Software Foundation Blog: http://srinathsview.blogspot.com/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
