Hi Kasun, We ran several rounds of load tests for ESB analytics (covering few scenarios with several proxy services, APIs, Sequences, etc). In the first round, we ran the tests with a low TPS (around 10-15) to test and validate the use cases, such as the counts, delays, drilling down to mediator/sequence/message level, etc. These scenarios worked as expected.
On the second round, we ran the same tests with a higher TPS (gradually increasing). In the process, we identified a bottleneck at the DB level under a high TPS (TPS of over 50, which is not that high), resulting the stat generation to be slowed down with time. This occurs as DAS is both reading and writing from/to the same table (row events) at the same time, at a rapid rate, causing the IO operations to be slowed down (due to table locks and etc.), and hence the summarizing process takes longer time, in each iteration. (most of the consumed time for summarizing, around 80% is taken for reading the raw data and store them as event-wise data in DB) Therefore, we are currently doing an enhancement to improve performance, by summarizing the data (summarize up to per-second info) on the fly with siddhi and store it, rather than spark doing it as a batch operation. This should eliminate roughly around 80% (above mentioned) of time taken to read/write row data from DB. Will give an update after re-running the load test with this enhancement. Regards, Supun On Mon, Apr 18, 2016 at 11:40 AM, Kasun Indrasiri <[email protected]> wrote: > Any updates on this? > > We must test the ESB analytics/ ESB 5 alpha release with the common > production use cases + high load. > > On Mon, Mar 28, 2016 at 2:38 PM, Buddhima Wijeweera <[email protected]> > wrote: > >> Hi All, >> >> We are currently in the process of testing statistics & tracing for >> various scenarios. >> Initially we planned to cover on the areas we have identified for >> functionality testing. >> Then we planned to move on to common scenarios, and performance testing >> as well. >> >> Thank you, >> >> On Mon, Mar 28, 2016 at 2:34 PM, Srinath Perera <[email protected]> wrote: >> >>> Please run ESB benchmark, get the events comes out of it to DAS, and use >>> that for testing. >>> >>> --Srinath >>> >>> On Mon, Mar 28, 2016 at 10:02 AM, Kasun Indrasiri <[email protected]> >>> wrote: >>> >>>> Hi Buddhima/VirajS, >>>> >>>> Once the end to end functionalities are implemented on the ESB >>>> analytics side, can we start testing the current implementation for >>>> production use cases of ESB. Basically we need to cover all the common >>>> integration scenarios and stress test the implemented with high load too. >>>> >>>> Thanks, >>>> Kasun >>>> >>>> -- >>>> Kasun Indrasiri >>>> Software Architect >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> cell: +94 77 556 5206 >>>> Blog : http://kasunpanorama.blogspot.com/ >>>> >>> >>> >>> >>> -- >>> ============================ >>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>> Site: http://home.apache.org/~hemapani/ >>> Photos: http://www.flickr.com/photos/hemapani/ >>> Phone: 0772360902 >>> >> >> >> >> -- >> Buddhima Wijeweera >> Software Engineer; WSO2 Inc.; http://wso2.com , >> >> Mobile: +94 71 427 9966 >> Email: [email protected] >> Blog: https://buddhimawijeweera.wordpress.com >> GitHub Profile: https://github.com/Buddhima >> > > > > -- > Kasun Indrasiri > Software Architect > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > cell: +94 77 556 5206 > Blog : http://kasunpanorama.blogspot.com/ > -- *Supun Sethunga* Software Engineer WSO2, Inc. http://wso2.com/ lean | enterprise | middleware Mobile : +94 716546324
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
