+1 for persisting through a single (elected?) node, and let Hazlecast do the replication.
I took into consideration the need to persist periodically instead of at each and every request (by spawning a separate thread that has access to the callerContext map)... so yes... we should think in the same way for replicating the counters across the cluster as well. Instead of using a global counter, can we perhaps use the last updated timestamp of each CallerContext? It's actually not a single counter we need to deal with, and each CallerContext instance will have separate counters mapped to their throttling policy AFAIK. Therefore, I think its probably better to update CallerContext instances based on the last update timestamp. WDYT? If agree, then I need to figure out how to make delayed replication on hazlecast (is it through the hazelcast.heartbeat.interval.seconds config item?) Regards, Manoj On Tue, Dec 17, 2013 at 4:22 PM, Srinath Perera <[email protected]> wrote: > We need to think it a cluster setup do we need persistence as well? As we > can have replication using Hazelcast? > > If we need persistence, I think it is a good if a single node persists the > current throttling values, and if that node fails, someone else takes it > place? > > Current implementation sync the values across the cluster per each > message, which introduce significant overhead. I think we should go to a > model where each node collects and update the values once few seconds. > > idea is > 1) there is a global counter, that we use to throttle > 2) Each node keep a global counter, and periodically it update the global > counter using value in location counter and reset the counter and read the > current global counter. > 3) Until next update, each node make decisions based on local global > counter values it has read already > > This will mean that the throttling will throttle close to the limit, not > exactly at the limit. However, IMHO, that is not a problem for throttling > usecase. > > --Srinath > > > > > On Mon, Dec 16, 2013 at 7:20 PM, Manoj Fernando <[email protected]> wrote: > >> Attaching Gdoc as a pdf. >> >> - Manoj >> >> >> On Mon, Dec 16, 2013 at 9:15 AM, Manoj Fernando <[email protected]> wrote: >> >>> All, >>> >>> We have a requirement for $subject. Like to hear your thoughts first on >>> the following plan, and setup a review session accordingly. >>> >>> Google doc @ [1] with permissions to comment. >>> >>> *Background* >>> Throttling is a core carbon component that provides API throttling >>> across the platform. The current implementation supports Role and >>> Concurrency based throttling which is used by products for more business >>> specific use cases. For example, the APIM uses the throttling framework to >>> provide throttling support at 3 levels. >>> >>> - Application Level - Policy is applied to the whole Application >>> (overrides any policy violations at the other 2 levels) >>> - API Level - Policy is applied at each API level (overrides any >>> policy violations at API Resource level) >>> - API Resource Level - Policy is applied at each API resource (i.e. >>> GET, POST, etc.) >>> >>> >>> *Problem* >>> At present, the core carbon framework does not persist the runtime >>> throttling data. For example, a role based APIM throttling policy may >>> specify that 50 requests be handled per minute, and if the APIM gateway >>> crashes at the 50th second having served 40 requests, a restart will cause >>> in APIM providing the full quota once the node is restarted. >>> >>> >>> *Current Design* >>> >>> >>> >>> >>> - ThrottleContext is initialized by APIThrottleHandler (in the case >>> of API Manager) at the time of the first authenticated request hitting >>> the >>> gateway. >>> - The APIThrottleHandler uses the ThrottleFactory (carbon core >>> class) to instantiate a ThrottleContext object. >>> - ThrottleContext keeps a map of CallerContext objects of which the >>> runtime throttle counters are kept, corresponding to each policy >>> definition >>> (e.g. A throttle scenario mapping the tier policy ‘Gold’ will initiate a >>> CallerContext at the first instance of the policy is matched.) >>> - For every new CallerContext instance, the ThrottleContext will >>> push that CallerContext instance to a Map. >>> - ThrottleContext exposes the ‘addCallerContext’ and >>> ‘removeCallerContext’ methods to add and to cleanup the expired context >>> objects. >>> - CallerContext keeps the caller count, and access times related to >>> the Caller. >>> - In the case of API Manager, each caller instance (based on the >>> tier configuration), access the ThrottleContext using the >>> doRoleBasedAccessThrottling and doThrottleByConcurrency methods. >>> >>> >>> *Implementing Persistence* >>> >>> >>> - ThrottleContext is independently initialized by any component >>> using the throttling framework. >>> - What needs to be persisted is the CallerContext map together with >>> the initiator attributes (i.e. TrottleID) >>> - An option is to spawn a separate Thread on the ThrottleContext >>> constructor that will have access to the CallerContext map. >>> - A new Persistence DAO (i.e. ThrottleContextPersister class), can >>> access the cached CallerContext instances using the >>> ThrottleUtil.getThrottleCache(). >>> - This ThrottleContextPersister needs to clean up the old caller >>> contexts entries on the DB before persisting the new caller entries. >>> - Persistence interval can be made configurable (carbon.xml ?). >>> >>> >>> >>> >>> >>> *Q&A* >>> >>> 1. How does this work on a clustered environment? >>> Irrespective of the node running on a cluster or not, we need to persist >>> the CallerContext map. >>> >>> Option A : Persist the caller context on an elected node in the cluster >>> given the fact that we can use Hazelcast to distribute the callercontext >>> map across the cluster nodes. >>> >>> Option B : Each node to independently persist their caller maps against >>> the node info. In this way, we will not have to rely on cluster >>> replication of the caller context map. >>> >>> 2. How does DB persistence done at the carbon core level? >>> [TODO : Find out how persistence is handled at the carbon core level.] >>> >>> 3. Are there any product specific objects that need to be persisted as >>> well? >>> AFAIK, no we do not need to. If you take the APIM for example, the tier >>> config gets loaded at the server startup and using the tier IDs we should >>> be able to initialize (load) the CallerContext map corresponding to that >>> scenario. >>> >>> 4. How often the CallerContext map need to be persisted? >>> As a thought, we should persist the CallerContext every 5-10 seconds >>> (IMO this should be a medium prio thread). Can we make this value >>> configurable? >>> >>> 5. Any chance of losing most recent runtime throttle info as we are not >>> persisting on each request? >>> Yes there is. But this is a trade-off between performance and the >>> requirement to persist throttle conters. Making the throttle persistence >>> interval configurable is a measure to control this. >>> >>> >>> 6. What needs to be persisted? >>> The following at a minimum >>> >>> ID : string /* The Id of caller */ >>> nextAccessTime : long /* next access time - the end of prohibition >>> */ >>> firstAccessTime : long /* when caller came across the on first time */ >>> nextTimeWindow : long /* beginning of next unit time period */ >>> count : int /* number of requests */ >>> >>> If we opt to use Option B for handling throttle persistence in a cluster >>> we will have to persist the nodeID in addition to these. >>> >>> >>> >>> [1] >>> https://docs.google.com/a/wso2.com/document/d/1AQOH-23jM37vjtzqoWg7vokUTsyaWh3eJLLoYQXYlf0 >>> >>> >>> Thoughts? >>> >>> Regards, >>> Manoj >>> -- >>> Manoj Fernando >>> Director - Solutions Architecture >>> >>> Contact: >>> LK - +94 112 145345 >>> Mob: +94 773 759340 >>> www.wso2.com >>> >> >> >> >> -- >> Manoj Fernando >> Director - Solutions Architecture >> >> Contact: >> LK - +94 112 145345 >> Mob: +94 773 759340 >> www.wso2.com >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Manoj Fernando Director - Solutions Architecture Contact: LK - +94 112 145345 Mob: +94 773 759340 www.wso2.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
