> > Persistence could be a case here. When we send events to CEP, it only > processes them in-memory and forgets about it. But in the case of DAS it > needs to persist each and every event it receives in the EVENT tables. This > could be taking up cycles and hence affecting data receival. >
I also agree with Nuwan. We did a similar performance test for IS analytics[1] and we observed that performance with persistence the is several factors less than the performance without persistence. Since we persist each incoming event into DB there's big impact on performance. [1] - [Architecture] IS-Analytics performance Thanks *,Sajith Ravindra* Senior Software Engineer WSO2 Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 77 2273550 blog: http://sajithr.blogspot.com/ <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> On Sun, Aug 21, 2016 at 11:27 AM, Imesh Gunaratne <[email protected]> wrote: > > > On Fri, Aug 19, 2016 at 2:23 PM, Supun Sethunga <[email protected]> wrote: > >> Hi all, >> >> Please find the deployment setup details at [1]. Some parts of it are yet >> to be completed though. >> >> @Imesh: No, we haven't created any puppet scripts for the setup. >> > > Thanks Supun! Few things I noticed: > > - > As I see we have run this on AWS. If so we can create a Cloud > Formation script to automate the entire deployment (may be Puppet can be > internally used but not mandatory). > - The reason why we need automation for this is, as I believe > performance tests should be able to run by any user for verifying the > results. We have done this in MSF4J. > > - Is there any reason to use a VM for MySQL without using RDS? > > T > hanks > > > >> >> @MalithJ: Input was controlled using Jmeter. Throughput is measured at >> the receiver side, in databridge level. (We can get it using >> -DprofileReceiver=true property) >> >> @Sanjeew/Harsha: This was tested with the default protocols (Binary for >> throttle publisher and thrift for the other publishers), but with tuned up >> data-agent-configs. Here I used four separate standalone APIm nodes, only >> to generate the events. This was not possible to simulate with a thrift >> client like we did for ESB Analytics, because here we have multiple >> receivers (unlike in ESB Analytics case), and the events are received at >> differents rates to those different receivers. We can publish to the >> streams separately, but it would not be same as a real world scenario.. >> Here the limitation of the throughput is caused by the complexity of the >> execution plans and the stream persisting, at the Analytics Server. >> >> [1] https://docs.google.com/document/d/1A1Pq-nOi9PmGdZz3wNov >> Pc7kG3vc74p93tOTh_WYp_o/edit# >> >> Regards, >> Supun >> >> On Fri, Aug 19, 2016 at 1:55 PM, Sanjeewa Malalgoda <[email protected]> >> wrote: >> >>> >>> >>> On Tue, Aug 16, 2016 at 10:43 PM, Harsha Kumara <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Tue, Aug 16, 2016 at 5:29 PM, Supun Sethunga <[email protected]> >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> We carried out some performance tests for APIM Analytics server in a >>>>> Minimum HA setup. Idea was to saturate the Analytics server up to the >>>>> maximum number of requests it can handle, without APIM (client side) >>>>> getting their publisher queues full, see how it will behave in the long >>>>> run >>>>> (for couple of hours). With this test, found out that the maximum >>>>> throughput it can handle is around 18,000 requests / sec. Please see >>>>> below. >>>>> For this test, used tomcat echo service as the backend. Used 4 standalone >>>>> APIM servers to achieve the throughput. >>>>> >>>> This means if we scale up the API gateways, it will lead to getting >>>> queue fill messages frequently when request throughput to DAS node is >>>> larger than 19000 TPS. Since DAS node can handle more than 19000 TPS, this >>>> behavior may cause due to rate of DAS server accepting messages is less >>>> than the rate of message coming to gateways which leads to filling the >>>> queues in gateways. Is this tested with binary or thrift transport? May be >>>> better if we can have results for both binary and thrift transports. >>>> >>> When we test API Manager throttling we sent throttling events to CEP >>> data receiver, we were able to send much more requests than this. >>> I think in DAS also we are using same data receiver implementation(like >>> harsha mentioned is it on thrift or binary?). >>> If i remember correctly single data receiver handled 30000 or more >>> requests(we had 10 gateways or so). >>> I think here we need to check how data receiver and analyzer behaves >>> when it have high load. So we may directly test it with client tool. If you >>> test with gateway then anyway it will fail at 4500TPS(even if Que filling >>> issue sorted) >>> >>> Thanks, >>> sanjeewa. >>> >>>> >>>>> Raw data can be found at [1]. >>>>> >>>>> [image: Inline image 1] >>>>> If we increase the throughput beyond 19,000 requests/sec, then the >>>>> queues at APIM side get filled after sometime. Please refer the 2nd, 3rd, >>>>> and 4th sheets of [1]. The huge variance towards the end in those charts >>>>> were due the queues getting filled. >>>>> >>>>> I will add these to a performance results template, including the >>>>> advanced details about the setups and the configs. >>>>> >>>>> [1] https://docs.google.com/spreadsheets/d/1vVRPp8b31DccJ1A6 >>>>> p9MRGxzmLAX2yiGAebT25kpsLzg/edit#gid=0 >>>>> >>>>> Regards, >>>>> Supun >>>>> >>>>> -- >>>>> *Supun Sethunga* >>>>> Senior Software Engineer >>>>> WSO2, Inc. >>>>> http://wso2.com/ >>>>> lean | enterprise | middleware >>>>> Mobile : +94 716546324 >>>>> Blog: http://supunsetunga.blogspot.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Harsha Kumara >>>> Software Engineer, WSO2 Inc. >>>> Mobile: +94775505618 >>>> Blog:harshcreationz.blogspot.com >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> >>> *Sanjeewa Malalgoda* >>> WSO2 Inc. >>> Mobile : +94713068779 >>> >>> <http://sanjeewamalalgoda.blogspot.com/>blog >>> :http://sanjeewamalalgoda.blogspot.com/ >>> <http://sanjeewamalalgoda.blogspot.com/> >>> >>> >>> >> >> >> -- >> *Supun Sethunga* >> Senior Software Engineer >> WSO2, Inc. >> http://wso2.com/ >> lean | enterprise | middleware >> Mobile : +94 716546324 >> Blog: http://supunsetunga.blogspot.com >> > > > > -- > *Imesh Gunaratne* > Software Architect > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: https://medium.com/@imesh TW: @imesh > lean. enterprise. middleware > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
