On 15 August 2012 08:05, Milamber <[email protected]> wrote: > > > Le 14/08/2012 21:10, sebb a ecrit : > >> On 14 August 2012 11:32, sebb<[email protected]> wrote: >>> >>> On 14 August 2012 10:51, Milamber<[email protected]> wrote: >>>> >>>> >>>> Le 13/08/2012 18:58, sebb a ecrit : >>>> >>>>> On 13 August 2012 17:07, sebb<[email protected]> wrote: >>>>>> >>>>>> I've made the changes to merge the OnDemand code back into the >>>>>> standard ThreadGroup. >>>>>> >>>>>> This means it's now trivial for users to switch between the different >>>>>> startup types. >>>>>> >>>>>> I hope I've not broken anything ... >>>>> >>>>> Seems to work OK with a simple test plan of a single Java sampler, 10 >>>>> loops. >>>>> Setting the ramp-up to the same as the thread count gives me a max of >>>>> 3 threads concurrently. >>>>> >>>>> This runs out of memory when run without delayed start and 10000 >>>>> threads. >>>>> The same test starts fine and runs for a while with delayed start. >>>> >>>> >>>> I've run a simple test of 2 Java sampler (with 1s sleep time), 10 loops, >>>> with 120000 threads and ramp-up 6000s with jmeter-trunk (on Linux 64 >>>> bits >>>> server with 48 cores, sun jdk 6 64bits) >>>> >>>> === >>>> With delayed start, test works fine. >>>> 2400004 of sampler results >>>> >>>> Max heap size during tests: ~2.6 GB >>>> (jmeter java opts : HEAP="-Xms8g -Xmx8g -XX:+UseConcMarkSweepGC >>>> -Xloggc:/tmp/gclog_`date '+20%y%m%d%H%M%S'`.txt") >>>> >>>> Average number of LWP (Light-weight process): ~500 (~threads inside the >>>> java process via the linux command 'ps') >>>> >>>> === >>>> Without delayed start: OutOfMemoryError: unable to create new native >>>> thread. >>>> 45176 of sampler results (test not ended, kill by me after the OOME >>>> notification) >>>> >>>> Max heap size during tests: ~2.3 GB >>>> (jmeter java opts : HEAP="-Xms8g -Xmx8g -XX:+UseConcMarkSweepGC >>>> -Xloggc:/tmp/gclog_`date '+20%y%m%d%H%M%S'`.txt") >>>> >>>> Max number of LWP (Light-weight process): 30396 (~threads inside the >>>> java >>>> process) >>>> >>>> === >>>> >>>> Conclusion IMHO: A great behavior for JMeter and tests with a huge >>>> number of >>>> threads (users) with small number of individual loops per user, like >>>> webservice's test >>> >>> Thanks! >>> >>> I'm currently working on delaying the creation of JMeterThread instances. >>> These take quite a bit of memory, and they aren't needed until just >>> before the thread starts. >>> >>> This means moving code from StandardJMeterEngine to ThreadGroup. >>> Quite a lot of changes, but it's looking very promising - many more >>> threads can be started, as memory is then only taken for active >>> threads. >>> >>> I should be able to commit the code later today or tomorrow. >> >> Committed. Seems to be working OK, though I've not tested multiple >> thread groups yet. >> >> When using delayed start, the thread count should now be limited only >> by the number of concurrent threads. >> I was able to start a test with 10 million threads (though I did not >> let it run for very long). >> >> Assuming that the thread-related data is properly released when the >> thread finishes, there's no reason why such a test should not >> complete. > > > > I've run again the simple test of 2 Java sampler (with 1s sleep time), 10 > loops, with 120000 threads and ramp-up 6000s with jmeter-trunk (on Linux 64 > bits server with 48 cores, sun jdk 6 64bits) > > > === > With delayed start, test works fine. > 2400004 of sampler results > > Max heap size during tests: end with 2.3 GB (without any Full GC) > > (jmeter java opts : HEAP="-Xms8g -Xmx8g -XX:+UseConcMarkSweepGC > -Xloggc:/tmp/gclog_`date '+20%y%m%d%H%M%S'`.txt") > > Average number of LWP (Light-weight process): ~500 (~threads inside the > java process via the linux command 'ps') > > === > Without delayed start: OutOfMemoryError: unable to create new native thread.
That's to be expected. > 105704 of sampler results (test not ended, kill by me after the OOME > notification) > > Max heap size during tests: ~7.8 GB > > (jmeter java opts : HEAP="-Xms8g -Xmx8g -XX:+UseConcMarkSweepGC > -Xloggc:/tmp/gclog_`date '+20%y%m%d%H%M%S'`.txt") > > Max number of LWP (Light-weight process): 31083 (~threads inside the java > process) > > === > > A new test: > same scenario but sleep_time = 100 ms, 2 loops, with 9,600,000 threads and > ramp-up 7200s, delayed start (80000 new threads by minute) > > Test have OOME after 50 min > ~7,715,164 of sampler results > > Max heap size during tests: end with 99 GB (after GC) (big OOME ;-)) > (jmeter java opts : HEAP="-Xms100g -Xmx100g -XX:NewSize=50g > -XX:MaxNewSize=50g -XX:PermSize=1g -XX:MaxPermSize=20g > -XX:+UseConcMarkSweepGC -Xloggc:/tmp/gclog_`date '+20%y%m%d%H%M%S'`.txt") > > Average number of LWP (Light-weight process): up to 769 (before the first > Full OOME) (~threads inside the java process via the linux command 'ps') This is a higher concurrent number than before. It would be interesting to know if the test would succeed with a similar count to before. i.e. keep the ratio of thread-count / ramp-up the same. That should show if there are thread resources that are not being cleaned up when the thread finishes. > Max TG number before the 1st OOME : Thread started: Thread Group 1-1875217 > > === > Milamber > > > > > >>>> Milamber >>>> >>>> >>>>> It's much quicker to stop the test as well. >>>>> >>>>> I was able to start a test with 50000 threads, but after that memory >>>>> was a problem. >>>>> Also, the test takes quite a long time to start up, presumably because >>>>> of the array of JMeterThread instances. >>>>> >>>>>> There's probably still some tidying up that could be done. >>>>>> >>>>>> For example, StandardJMeterEngine still creates the JMeterThread >>>>>> instances at the start. It might be better to make the thread group >>>>>> class responsible for handling these, so the additional objects are >>>>>> not created until needed. >>>>> >>>>> I think that's what broke my tests with more than about 60000 threads. >>>>> >>>>> As well as deferring creation of these objects,they need to be >>>>> released once the thread has completed. >>>>> >>>>>> BTW, my French is not up to translating the delayed_start property >>>>>> entry - please can someone else fix that so the tests no longer fail? >>>>>> Thanks! >>>> >>>> >
