Hi Nuwan, Understood your concern Nuwan, I'm the one who proposed to add peak load for few hours( ~ 4 hours) and check the memory growth, GC , exceptions etc in Gateway and KM sides. In here we are not trying to hit the server with max load, just want to check the server behavior. :) If you are not agreeing to this, will do proper long running test for 2 weeks with load that you suggested and will share the results with you. Then 3rd week will do some ad-hoc test like we suggested and check server behavior. WDYT?
Regards, Aparna. On Fri, Jan 8, 2016 at 11:01 AM, Nuwan Dias <[email protected]> wrote: > Hi Aparna, > > I'm totally +1 for the long running test. But my concern is that Pubudu is > planning for a strange test. It doesn't look like a typical long running > test. See his comment on the last reply "*generate a relatively higher > number of threads lets say for a few hours of the day and stop it on daily > basis*". The idea I get from this is that you're trying to hit the server > at max load and then stop... hit it at max again and stop... likewise. > Maybe I'm getting the wrong message but the information you've requested is > for the max threads count and not the moderate threads count. > > IMO to mimick a real world scenario you should be hitting the sever with a > moderate load (about 50 threads max) for a long time period. Not with a > peak load (250 threads) for several hours. And we should monitor the > servers during this period in order to have insight of any issues that may > occur. The closer we get to the real world occurrence, the more valuable > the outcome of our efforts would be. > > Thanks, > NuwanD. > > On Fri, Jan 8, 2016 at 10:47 AM, Aparna Karunarathna <[email protected]> > wrote: > >> Hi Nuwan, >> >> Objective of doing a long running test after the release is, in the >> release time we won't be able to execute a long running test for couple of >> weeks. Because sometime we have to stop the test to change the packs or >> integrate with other product with setup or to enable some configs to test a >> feature, etc. Most of the time we are getting some cloud issue as >> well(resource issues and set of nodes were getting slow). So most of time >> we were able to run the long running test for day or 2 days. >> >> Therefor we thought of doing a proper long running test against the GA >> pack for 3 weeks. Last year we did long running test after the release for >> couple of releases and then we shared our findings(memory growth, >> exceptions, warnings, etc) with dev teams. So if we can identify a bug at >> this stage then we can prepare a fix. >> >> Regards, >> Aparna. >> >> >> >> On Fri, Jan 8, 2016 at 9:22 AM, Nuwan Dias <[email protected]> wrote: >> >>> >>> >>> On Fri, Jan 8, 2016 at 7:03 AM, Pubudu Priyashan <[email protected]> >>> wrote: >>> >>>> Hi Nuwan, >>>> >>>> The idea is not to keep the peak level constant throughout the duration >>>> of the long running test but, to generate a relatively higher number of >>>> threads lets say for a few hours of the day and stop it on daily basis. >>>> >>> >>> What is the objective of this test? From what I understand, its not a >>> performance test and its not a typical long running test as well. So can >>> you explain what you're trying to achieve and how it benefits the product >>> or any other? When doing performance tests the peak thread count we hit was >>> around 250 and you can use it if you want. But its better to do it with a >>> plan and clear objective. >>> >>> >>>> Appreciate if you could suggest a number for the peak thread count per >>>> single gateway node so we can monitor the behaviour. >>>> >>>> Currently we don't have scripts to support the scenario you suggested. >>>> We will start working on this and create relevant scripts and update you >>>> once we have tested the scenario you have mentioned. Thanks. >>>> >>>> Cheers, >>>> Pubudu. >>>> >>>> Pubudu D.P >>>> Senior Software Engineer - QA Team | WSO2 inc. >>>> Mobile : +94775464547 >>>> >>>> On Thu, Jan 7, 2016 at 4:01 PM, Nuwan Dias <[email protected]> wrote: >>>> >>>>> Hi Pubudu, >>>>> >>>>> I don't think its useful to do a long running test at peak load >>>>> because it doesn't happen in reality. Any system can hit peaks, but it >>>>> would be a momentary occurrence rather than a long running situation. >>>>> >>>>> What would be more useful is to do the long running test under >>>>> moderate load. This would mimick many real world scenarios. Therefore I >>>>> would recommend to have the concurrency at 50 threads, using about 1000 >>>>> access tokens and run the test in such a way that it renews its access >>>>> tokens from time to time (either when they expire or once every 45 minutes >>>>> or so). >>>>> >>>>> We should also configure the Java Flight Recorder (remotely, to avoid >>>>> filling up the file system) so that we would have insight into issues if >>>>> any. Also please make sure that the back-end system you front through the >>>>> gateway is stable and can withhold the tests. >>>>> >>>>> Thanks, >>>>> NuwanD. >>>>> >>>>> On Thu, Jan 7, 2016 at 3:18 PM, Pubudu Priyashan <[email protected]> >>>>> wrote: >>>>> >>>>>> >>>>>> Hi Nuwan, >>>>>> >>>>>> We are planning to start a long running test on the API-M 1.10.0 >>>>>> released pack. As a part of this we need to simulate a peak time in >>>>>> gateway >>>>>> worker nodes (by sending a large number of concurrent threads) could you >>>>>> please let us know what is the maximum expected limit of concurrent >>>>>> threads >>>>>> that we can be served in a single gateway worker node? We will then set >>>>>> up >>>>>> the long-running tests accordingly. Thanks! >>>>>> >>>>>> Cheers, >>>>>> Pubudu D.P >>>>>> Senior Software Engineer - QA Team | WSO2 inc. >>>>>> Mobile : +94775464547 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Technical Lead - WSO2, Inc. http://wso2.com >>>>> email : [email protected] >>>>> Phone : +94 777 775 729 >>>>> >>>> >>>> >>> >>> >>> -- >>> Nuwan Dias >>> >>> Technical Lead - WSO2, Inc. http://wso2.com >>> email : [email protected] >>> Phone : +94 777 775 729 >>> >> >> >> >> -- >> *Regards,* >> >> *Aparna Karunarathna.* >> >> >> *Associate Technical Lead - QAWSO2 Inc.Mobile: 0714002533 <0714002533>* >> > > > > -- > Nuwan Dias > > Technical Lead - WSO2, Inc. http://wso2.com > email : [email protected] > Phone : +94 777 775 729 > -- *Regards,* *Aparna Karunarathna.* *Associate Technical Lead - QAWSO2 Inc.Mobile: 0714002533*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
