On Wed, Dec 18, 2013 at 9:39 AM, Manoj Fernando <[email protected]> wrote:
> Hi Azeez, > > hmmm true... I too came across quite a few 'canAccess' calls that I felt > could be simplified. However, this is certainly a bigger change than just > introing persistence. So how should we proceed from this point onwards? A > complete rewrite of the throttle core means that we will have to have > another broader look of the whole module. IMO, we can retain the ipbase, > rolebase, and domainbase APIs, yet will have to re-write the Controller > logic (i.e. ConcurrentAccessThrottler, and RoleBasedAccessRateThrottler + > helpers). > > Shall we start a new thread for a re-write then? > Yeah, +1. The original throttling code was written in 2006 & has not been used much in clustered deployments until recently. It would be best to reimplement it with all the requirements in mind. > Regards. > Manoj > > > On Tue, Dec 17, 2013 at 9:28 PM, Afkham Azeez <[email protected]> wrote: > >> Hi Manoj, >> While working on a critical state replication issue in throttling core, >> we figured that the code can be better written. Piling on changes on to >> that code will only make it worse, and even now the code looks >> unmaintainable. I would prefer redesigning & rewriting throttling core from >> scratch taking into consideration persistence & state replication. >> >> Azeez >> >> >> On Mon, Dec 16, 2013 at 9:15 AM, Manoj Fernando <[email protected]> wrote: >> >>> All, >>> >>> We have a requirement for $subject. Like to hear your thoughts first on >>> the following plan, and setup a review session accordingly. >>> >>> Google doc @ [1] with permissions to comment. >>> >>> *Background* >>> Throttling is a core carbon component that provides API throttling >>> across the platform. The current implementation supports Role and >>> Concurrency based throttling which is used by products for more business >>> specific use cases. For example, the APIM uses the throttling framework to >>> provide throttling support at 3 levels. >>> >>> - Application Level - Policy is applied to the whole Application >>> (overrides any policy violations at the other 2 levels) >>> - API Level - Policy is applied at each API level (overrides any >>> policy violations at API Resource level) >>> - API Resource Level - Policy is applied at each API resource (i.e. >>> GET, POST, etc.) >>> >>> >>> *Problem* >>> At present, the core carbon framework does not persist the runtime >>> throttling data. For example, a role based APIM throttling policy may >>> specify that 50 requests be handled per minute, and if the APIM gateway >>> crashes at the 50th second having served 40 requests, a restart will cause >>> in APIM providing the full quota once the node is restarted. >>> >>> >>> *Current Design* >>> >>> >>> >>> >>> - ThrottleContext is initialized by APIThrottleHandler (in the case >>> of API Manager) at the time of the first authenticated request hitting >>> the >>> gateway. >>> - The APIThrottleHandler uses the ThrottleFactory (carbon core >>> class) to instantiate a ThrottleContext object. >>> - ThrottleContext keeps a map of CallerContext objects of which the >>> runtime throttle counters are kept, corresponding to each policy >>> definition >>> (e.g. A throttle scenario mapping the tier policy ‘Gold’ will initiate a >>> CallerContext at the first instance of the policy is matched.) >>> - For every new CallerContext instance, the ThrottleContext will >>> push that CallerContext instance to a Map. >>> - ThrottleContext exposes the ‘addCallerContext’ and >>> ‘removeCallerContext’ methods to add and to cleanup the expired context >>> objects. >>> - CallerContext keeps the caller count, and access times related to >>> the Caller. >>> - In the case of API Manager, each caller instance (based on the >>> tier configuration), access the ThrottleContext using the >>> doRoleBasedAccessThrottling and doThrottleByConcurrency methods. >>> >>> >>> *Implementing Persistence* >>> >>> >>> - ThrottleContext is independently initialized by any component >>> using the throttling framework. >>> - What needs to be persisted is the CallerContext map together with >>> the initiator attributes (i.e. TrottleID) >>> - An option is to spawn a separate Thread on the ThrottleContext >>> constructor that will have access to the CallerContext map. >>> - A new Persistence DAO (i.e. ThrottleContextPersister class), can >>> access the cached CallerContext instances using the >>> ThrottleUtil.getThrottleCache(). >>> - This ThrottleContextPersister needs to clean up the old caller >>> contexts entries on the DB before persisting the new caller entries. >>> - Persistence interval can be made configurable (carbon.xml ?). >>> >>> >>> >>> >>> >>> *Q&A* >>> >>> 1. How does this work on a clustered environment? >>> Irrespective of the node running on a cluster or not, we need to persist >>> the CallerContext map. >>> >>> Option A : Persist the caller context on an elected node in the cluster >>> given the fact that we can use Hazelcast to distribute the callercontext >>> map across the cluster nodes. >>> >>> Option B : Each node to independently persist their caller maps against >>> the node info. In this way, we will not have to rely on cluster >>> replication of the caller context map. >>> >>> 2. How does DB persistence done at the carbon core level? >>> [TODO : Find out how persistence is handled at the carbon core level.] >>> >>> 3. Are there any product specific objects that need to be persisted as >>> well? >>> AFAIK, no we do not need to. If you take the APIM for example, the tier >>> config gets loaded at the server startup and using the tier IDs we should >>> be able to initialize (load) the CallerContext map corresponding to that >>> scenario. >>> >>> 4. How often the CallerContext map need to be persisted? >>> As a thought, we should persist the CallerContext every 5-10 seconds >>> (IMO this should be a medium prio thread). Can we make this value >>> configurable? >>> >>> 5. Any chance of losing most recent runtime throttle info as we are not >>> persisting on each request? >>> Yes there is. But this is a trade-off between performance and the >>> requirement to persist throttle conters. Making the throttle persistence >>> interval configurable is a measure to control this. >>> >>> >>> 6. What needs to be persisted? >>> The following at a minimum >>> >>> ID : string /* The Id of caller */ >>> nextAccessTime : long /* next access time - the end of prohibition >>> */ >>> firstAccessTime : long /* when caller came across the on first time */ >>> nextTimeWindow : long /* beginning of next unit time period */ >>> count : int /* number of requests */ >>> >>> If we opt to use Option B for handling throttle persistence in a cluster >>> we will have to persist the nodeID in addition to these. >>> >>> >>> >>> [1] >>> https://docs.google.com/a/wso2.com/document/d/1AQOH-23jM37vjtzqoWg7vokUTsyaWh3eJLLoYQXYlf0 >>> >>> >>> Thoughts? >>> >>> Regards, >>> Manoj >>> -- >>> Manoj Fernando >>> Director - Solutions Architecture >>> >>> Contact: >>> LK - +94 112 145345 >>> Mob: +94 773 759340 >>> www.wso2.com >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Manoj Fernando > Director - Solutions Architecture > > Contact: > LK - +94 112 145345 > Mob: +94 773 759340 > www.wso2.com > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919 blog: **http://blog.afkham.org*<http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> * linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
