Hi all, We are doing a performance test on IS analytics server. While we keep pumping data to the server we can see that throughput gradually decreases to a very low (< 1K) TPS. We used MySQL with InnoDB with tuned parameters. And set Xms2g and Xmx4g in DAS.
When looked at the thread dumps we can see that more than 100 threads BLOCKED/TIMED_WATING at AnalyticsEventQueueManager.put() on a single lock object(i.e. on the same reference). Many threads are just waiting to get the lock without processing events which seem to be the reason for degraded performance. Are there any parameters that we can tune to gain better performance to overcome this bottleneck ? I had a look at the code and AFAIU all events of a particular tenant are persisted through a single synchronized queue. According to my understanding, this synchronized queue is a performance bottleneck since all threads pertaining to a persisted event stream will need to block on this queue. IMO if we can have a queue per event-sink(i.e per persisted stream) it should give a better performance than this. Any implications if we do that or is there a better option? Thanks *,Sajith Ravindra* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273550 blog: http://sajithr.blogspot.com/ <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
