On Fri, Aug 19, 2016 at 2:23 PM, Supun Sethunga <[email protected]> wrote:
> Hi all, > > Please find the deployment setup details at [1]. Some parts of it are yet > to be completed though. > > @Imesh: No, we haven't created any puppet scripts for the setup. > Thanks Supun! Few things I noticed: - As I see we have run this on AWS. If so we can create a Cloud Formation script to automate the entire deployment (may be Puppet can be internally used but not mandatory). - The reason why we need automation for this is, as I believe performance tests should be able to run by any user for verifying the results. We have done this in MSF4J. - Is there any reason to use a VM for MySQL without using RDS? T hanks > > @MalithJ: Input was controlled using Jmeter. Throughput is measured at the > receiver side, in databridge level. (We can get it using > -DprofileReceiver=true property) > > @Sanjeew/Harsha: This was tested with the default protocols (Binary for > throttle publisher and thrift for the other publishers), but with tuned up > data-agent-configs. Here I used four separate standalone APIm nodes, only > to generate the events. This was not possible to simulate with a thrift > client like we did for ESB Analytics, because here we have multiple > receivers (unlike in ESB Analytics case), and the events are received at > differents rates to those different receivers. We can publish to the > streams separately, but it would not be same as a real world scenario.. > Here the limitation of the throughput is caused by the complexity of the > execution plans and the stream persisting, at the Analytics Server. > > [1] https://docs.google.com/document/d/1A1Pq- > nOi9PmGdZz3wNovPc7kG3vc74p93tOTh_WYp_o/edit# > > Regards, > Supun > > On Fri, Aug 19, 2016 at 1:55 PM, Sanjeewa Malalgoda <[email protected]> > wrote: > >> >> >> On Tue, Aug 16, 2016 at 10:43 PM, Harsha Kumara <[email protected]> wrote: >> >>> >>> >>> On Tue, Aug 16, 2016 at 5:29 PM, Supun Sethunga <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> We carried out some performance tests for APIM Analytics server in a >>>> Minimum HA setup. Idea was to saturate the Analytics server up to the >>>> maximum number of requests it can handle, without APIM (client side) >>>> getting their publisher queues full, see how it will behave in the long run >>>> (for couple of hours). With this test, found out that the maximum >>>> throughput it can handle is around 18,000 requests / sec. Please see below. >>>> For this test, used tomcat echo service as the backend. Used 4 standalone >>>> APIM servers to achieve the throughput. >>>> >>> This means if we scale up the API gateways, it will lead to getting >>> queue fill messages frequently when request throughput to DAS node is >>> larger than 19000 TPS. Since DAS node can handle more than 19000 TPS, this >>> behavior may cause due to rate of DAS server accepting messages is less >>> than the rate of message coming to gateways which leads to filling the >>> queues in gateways. Is this tested with binary or thrift transport? May be >>> better if we can have results for both binary and thrift transports. >>> >> When we test API Manager throttling we sent throttling events to CEP data >> receiver, we were able to send much more requests than this. >> I think in DAS also we are using same data receiver implementation(like >> harsha mentioned is it on thrift or binary?). >> If i remember correctly single data receiver handled 30000 or more >> requests(we had 10 gateways or so). >> I think here we need to check how data receiver and analyzer behaves when >> it have high load. So we may directly test it with client tool. If you test >> with gateway then anyway it will fail at 4500TPS(even if Que filling issue >> sorted) >> >> Thanks, >> sanjeewa. >> >>> >>>> Raw data can be found at [1]. >>>> >>>> [image: Inline image 1] >>>> If we increase the throughput beyond 19,000 requests/sec, then the >>>> queues at APIM side get filled after sometime. Please refer the 2nd, 3rd, >>>> and 4th sheets of [1]. The huge variance towards the end in those charts >>>> were due the queues getting filled. >>>> >>>> I will add these to a performance results template, including the >>>> advanced details about the setups and the configs. >>>> >>>> [1] https://docs.google.com/spreadsheets/d/1vVRPp8b31DccJ1A6 >>>> p9MRGxzmLAX2yiGAebT25kpsLzg/edit#gid=0 >>>> >>>> Regards, >>>> Supun >>>> >>>> -- >>>> *Supun Sethunga* >>>> Senior Software Engineer >>>> WSO2, Inc. >>>> http://wso2.com/ >>>> lean | enterprise | middleware >>>> Mobile : +94 716546324 >>>> Blog: http://supunsetunga.blogspot.com >>>> >>> >>> >>> >>> -- >>> Harsha Kumara >>> Software Engineer, WSO2 Inc. >>> Mobile: +94775505618 >>> Blog:harshcreationz.blogspot.com >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> *Sanjeewa Malalgoda* >> WSO2 Inc. >> Mobile : +94713068779 >> >> <http://sanjeewamalalgoda.blogspot.com/>blog >> :http://sanjeewamalalgoda.blogspot.com/ >> <http://sanjeewamalalgoda.blogspot.com/> >> >> >> > > > -- > *Supun Sethunga* > Senior Software Engineer > WSO2, Inc. > http://wso2.com/ > lean | enterprise | middleware > Mobile : +94 716546324 > Blog: http://supunsetunga.blogspot.com > -- *Imesh Gunaratne* Software Architect WSO2 Inc: http://wso2.com T: +94 11 214 5345 M: +94 77 374 2057 W: https://medium.com/@imesh TW: @imesh lean. enterprise. middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
