Hi Srinath, That sounds like a much cleaner solution. We can perhaps use the native map-store declarative [1] which I think does something similar. It may sound a little silly to ask... but are we keeping Hazlecast active on a single node environment as well? :) Otherwise we will have to handle persistence on a single node in a different way. This is with the assumption of needing to persist throttle data on a single node environment as well (but questioning if we really need to do that is not totally invalid IMO).
Shall we go ahead with the Hazlecast option targeting cluster deployments then? - Manoj [1] https://code.google.com/p/hazelcast/wiki/MapPersistence On Thu, Dec 19, 2013 at 10:51 AM, Srinath Perera <[email protected]> wrote: > One another way to do this use Hazelcast and then use "though cache" or > "Change listener's" in Hazecast for persistence. > > --Srinath > > > On Tue, Dec 17, 2013 at 4:49 PM, Manoj Fernando <[email protected]> wrote: > >> +1 for persisting through a single (elected?) node, and let Hazlecast do >> the replication. >> >> I took into consideration the need to persist periodically instead of at >> each and every request (by spawning a separate thread that has access to >> the callerContext map)... so yes... we should think in the same way for >> replicating the counters across the cluster as well. >> >> Instead of using a global counter, can we perhaps use the last updated >> timestamp of each CallerContext? It's actually not a single counter we >> need to deal with, and each CallerContext instance will have separate >> counters mapped to their throttling policy AFAIK. Therefore, I think its >> probably better to update CallerContext instances based on the last update >> timestamp. >> >> WDYT? >> >> If agree, then I need to figure out how to make delayed replication on >> hazlecast (is it through the hazelcast.heartbeat.interval.seconds config >> item?) >> >> Regards, >> Manoj >> >> >> On Tue, Dec 17, 2013 at 4:22 PM, Srinath Perera <[email protected]> wrote: >> >>> We need to think it a cluster setup do we need persistence as well? As >>> we can have replication using Hazelcast? >>> >>> If we need persistence, I think it is a good if a single node persists >>> the current throttling values, and if that node fails, someone else takes >>> it place? >>> >>> Current implementation sync the values across the cluster per each >>> message, which introduce significant overhead. I think we should go to a >>> model where each node collects and update the values once few seconds. >>> >>> idea is >>> 1) there is a global counter, that we use to throttle >>> 2) Each node keep a global counter, and periodically it update the >>> global counter using value in location counter and reset the counter and >>> read the current global counter. >>> 3) Until next update, each node make decisions based on local global >>> counter values it has read already >>> >>> This will mean that the throttling will throttle close to the limit, not >>> exactly at the limit. However, IMHO, that is not a problem for throttling >>> usecase. >>> >>> --Srinath >>> >>> >>> >>> >>> On Mon, Dec 16, 2013 at 7:20 PM, Manoj Fernando <[email protected]> wrote: >>> >>>> Attaching Gdoc as a pdf. >>>> >>>> - Manoj >>>> >>>> >>>> On Mon, Dec 16, 2013 at 9:15 AM, Manoj Fernando <[email protected]>wrote: >>>> >>>>> All, >>>>> >>>>> We have a requirement for $subject. Like to hear your thoughts first >>>>> on the following plan, and setup a review session accordingly. >>>>> >>>>> Google doc @ [1] with permissions to comment. >>>>> >>>>> *Background* >>>>> Throttling is a core carbon component that provides API throttling >>>>> across the platform. The current implementation supports Role and >>>>> Concurrency based throttling which is used by products for more business >>>>> specific use cases. For example, the APIM uses the throttling framework >>>>> to >>>>> provide throttling support at 3 levels. >>>>> >>>>> - Application Level - Policy is applied to the whole Application >>>>> (overrides any policy violations at the other 2 levels) >>>>> - API Level - Policy is applied at each API level (overrides any >>>>> policy violations at API Resource level) >>>>> - API Resource Level - Policy is applied at each API resource >>>>> (i.e. GET, POST, etc.) >>>>> >>>>> >>>>> *Problem* >>>>> At present, the core carbon framework does not persist the runtime >>>>> throttling data. For example, a role based APIM throttling policy may >>>>> specify that 50 requests be handled per minute, and if the APIM gateway >>>>> crashes at the 50th second having served 40 requests, a restart will cause >>>>> in APIM providing the full quota once the node is restarted. >>>>> >>>>> >>>>> *Current Design* >>>>> >>>>> >>>>> >>>>> >>>>> - ThrottleContext is initialized by APIThrottleHandler (in the >>>>> case of API Manager) at the time of the first authenticated request >>>>> hitting >>>>> the gateway. >>>>> - The APIThrottleHandler uses the ThrottleFactory (carbon core >>>>> class) to instantiate a ThrottleContext object. >>>>> - ThrottleContext keeps a map of CallerContext objects of which >>>>> the runtime throttle counters are kept, corresponding to each policy >>>>> definition (e.g. A throttle scenario mapping the tier policy ‘Gold’ >>>>> will >>>>> initiate a CallerContext at the first instance of the policy is >>>>> matched.) >>>>> - For every new CallerContext instance, the ThrottleContext will >>>>> push that CallerContext instance to a Map. >>>>> - ThrottleContext exposes the ‘addCallerContext’ and >>>>> ‘removeCallerContext’ methods to add and to cleanup the expired context >>>>> objects. >>>>> - CallerContext keeps the caller count, and access times related >>>>> to the Caller. >>>>> - In the case of API Manager, each caller instance (based on the >>>>> tier configuration), access the ThrottleContext using the >>>>> doRoleBasedAccessThrottling and doThrottleByConcurrency methods. >>>>> >>>>> >>>>> *Implementing Persistence* >>>>> >>>>> >>>>> - ThrottleContext is independently initialized by any component >>>>> using the throttling framework. >>>>> - What needs to be persisted is the CallerContext map together >>>>> with the initiator attributes (i.e. TrottleID) >>>>> - An option is to spawn a separate Thread on the ThrottleContext >>>>> constructor that will have access to the CallerContext map. >>>>> - A new Persistence DAO (i.e. ThrottleContextPersister class), can >>>>> access the cached CallerContext instances using the >>>>> ThrottleUtil.getThrottleCache(). >>>>> - This ThrottleContextPersister needs to clean up the old caller >>>>> contexts entries on the DB before persisting the new caller entries. >>>>> - Persistence interval can be made configurable (carbon.xml ?). >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *Q&A* >>>>> >>>>> 1. How does this work on a clustered environment? >>>>> Irrespective of the node running on a cluster or not, we need to >>>>> persist the CallerContext map. >>>>> >>>>> Option A : Persist the caller context on an elected node in the >>>>> cluster given the fact that we can use Hazelcast to distribute the >>>>> callercontext map across the cluster nodes. >>>>> >>>>> Option B : Each node to independently persist their caller maps >>>>> against the node info. In this way, we will not have to rely on cluster >>>>> replication of the caller context map. >>>>> >>>>> 2. How does DB persistence done at the carbon core level? >>>>> [TODO : Find out how persistence is handled at the carbon core level.] >>>>> >>>>> 3. Are there any product specific objects that need to be persisted as >>>>> well? >>>>> AFAIK, no we do not need to. If you take the APIM for example, the >>>>> tier config gets loaded at the server startup and using the tier IDs we >>>>> should be able to initialize (load) the CallerContext map corresponding to >>>>> that scenario. >>>>> >>>>> 4. How often the CallerContext map need to be persisted? >>>>> As a thought, we should persist the CallerContext every 5-10 seconds >>>>> (IMO this should be a medium prio thread). Can we make this value >>>>> configurable? >>>>> >>>>> 5. Any chance of losing most recent runtime throttle info as we are >>>>> not persisting on each request? >>>>> Yes there is. But this is a trade-off between performance and the >>>>> requirement to persist throttle conters. Making the throttle persistence >>>>> interval configurable is a measure to control this. >>>>> >>>>> >>>>> 6. What needs to be persisted? >>>>> The following at a minimum >>>>> >>>>> ID : string /* The Id of caller */ >>>>> nextAccessTime : long /* next access time - the end of >>>>> prohibition */ >>>>> firstAccessTime : long /* when caller came across the on first time */ >>>>> nextTimeWindow : long /* beginning of next unit time period */ >>>>> count : int /* number of requests */ >>>>> >>>>> If we opt to use Option B for handling throttle persistence in a >>>>> cluster we will have to persist the nodeID in addition to these. >>>>> >>>>> >>>>> >>>>> [1] >>>>> https://docs.google.com/a/wso2.com/document/d/1AQOH-23jM37vjtzqoWg7vokUTsyaWh3eJLLoYQXYlf0 >>>>> >>>>> >>>>> Thoughts? >>>>> >>>>> Regards, >>>>> Manoj >>>>> -- >>>>> Manoj Fernando >>>>> Director - Solutions Architecture >>>>> >>>>> Contact: >>>>> LK - +94 112 145345 >>>>> Mob: +94 773 759340 >>>>> www.wso2.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Manoj Fernando >>>> Director - Solutions Architecture >>>> >>>> Contact: >>>> LK - +94 112 145345 >>>> Mob: +94 773 759340 >>>> www.wso2.com >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> ============================ >>> Srinath Perera, Ph.D. >>> http://people.apache.org/~hemapani/ >>> http://srinathsview.blogspot.com/ >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Manoj Fernando >> Director - Solutions Architecture >> >> Contact: >> LK - +94 112 145345 >> Mob: +94 773 759340 >> www.wso2.com >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > ============================ > Srinath Perera, Ph.D. > Director, Research, WSO2 Inc. > Visiting Faculty, University of Moratuwa > Member, Apache Software Foundation > Research Scientist, Lanka Software Foundation > Blog: http://srinathsview.blogspot.com/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Manoj Fernando Director - Solutions Architecture Contact: LK - +94 112 145345 Mob: +94 773 759340 www.wso2.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
