Hi Sanjeewa, I think increasing internal parameters would help when API calls are happening from lot of different tokens. In the test, I only invoked one API using one token which means that only one segment of the CHM, will be always accessed and all the threads will try to lock the same segment all the time. So I don't think, increasing concurrency Level would help at this point.
While reading about this issue, came across several articles mentioning that map.get() could return a null, if it's overlapped with a remove or an update operation. I tried reducing the number of places where values are written to map and now throttling out happens more consistently. But still there a lot of threads seeing null, when getting CallerContext from the map. On Fri, Nov 21, 2014 at 11:40 AM, Sanjeewa Malalgoda <[email protected]> wrote: > > > On Fri, Nov 21, 2014 at 10:23 AM, Amila De Silva <[email protected]> wrote: > >> Hi, >> >> I encountered the following problem while working on [1] >> >> As to provide some context on the problem. This problem is related to >> throttling API calls Gateway. In Gateway, throttle counts for API >> invocations are maintained by creating a CallerContext >> <https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/throttling/org.wso2.carbon.throttle.core/4.2.0/src/main/java/org/wso2/carbon/throttle/core/CallerContext.java> >> object for each unique accesstoken+apicontext+version combination. Such >> created CallerContexts are kept in a Concurrent HashMap to allow multiple >> threads read and update throttle counts. >> >> When a request passes through the Throttle Handler, it'll first get the >> correct CallerContext from the map and then increment it's count. If the >> key returns a null value,then a new CallerContext will be created and >> stored in the HashMap against the same key. It was written in this way >> assuming that null is only returned in the first invocation (return of null >> value is used to initialise the CallerContext). >> >> The problem is, under a considerable load, HashMap.get() returns a null >> despite the object being present. Since a new CallerContext is created and >> stored upon seeing a null value, the original object gets overwritten >> messing up throttling limits for the particular API. >> >> To prevent CallerContext being overwritten, HashMap.putIfAbsent() was >> used instead of put(). But still the problem with HashMap.get() being null >> couldn't be solved. Earlier there was one place where entries are removed >> from the map , but this is still present even after removing all the >> deletions. >> > IMO caller context should be overwritten for each and every call we made. > Otherwise counters will not update properly and will end up with messed up > throttle counters, time values etc. WDYT? > And i read that default concurrency level(shards) for concurrent hashmap > is 16 and sometimes number of synapse worker threads(number of writer > threads in this case to hash map) can be more than that. It seems this is > interesting issue and need to read bit. May be we can change internalizing > parameters (initialCapacity, loadFactor, concurrencyLevel) and run some > other tests. > > Thanks, > sanjeewa. > >> One possible way to overcome this is to synchronize all the puts and gets >> using the same key (which might degrade performance). >> >> Has anyone come across a similar issue? >> >> [1] https://wso2.org/jira/browse/APIMANAGER-3075 >> -- >> *Amila De Silva* >> >> WSO2 Inc. >> mobile :(+94) 775119302 >> >> > > > -- > > *Sanjeewa Malalgoda* > WSO2 Inc. > Mobile : +94713068779 > > <http://sanjeewamalalgoda.blogspot.com/>blog > :http://sanjeewamalalgoda.blogspot.com/ > <http://sanjeewamalalgoda.blogspot.com/> > > > -- *Amila De Silva* WSO2 Inc. mobile :(+94) 775119302
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
