I'm curious to know if the system under test is https
-----Original Message----- From: sebb <[email protected]> Date: Thu, 15 Dec 2011 00:35:39 To: JMeter Users List<[email protected]> Reply-To: "JMeter Users List" <[email protected]> Subject: Re: Strange 'pause' activity during testing On 15 December 2011 00:22, Robin D. Wilson <[email protected]> wrote: > I can use non-GUI mode for testing out the problem, but with the exact same > test - just switching the 'Retrieve All Embedded...' disabled, the problem > goes away. > > The only listeners I have enabled are the summary listener - the PerfMon > listeners (1 for each server) - and a 'tree' listener, but it only reports > 'errors'. > > I can run the test with 'Retrieve All Embedded...' enabled, I see the > problem every time. Maybe then there is a problem with the scanning of the HTML to extract the embedded resources, or maybe one of the embedded resources is a tar-pit. Do all the expected embedded resources get downloaded? Are there any unusually long elapsed times for embedded resources? Or large gaps between the parent sample download completion and the start of the first embedded resource? That would suggest the page took a while to parse. You'll need to select the optiion to "save subresults" in order to see the embedded samples. > If I just uncheck the box in the Request Defaults and re-run the test, it > never happens... So are you suggesting that I've hit some arbitrary limit of > the GUI mode? No, just that GUI mode is inherently more resource intensive. BTW, you wrote you were running JMeter 2.4.1 - that does not exist, so perhaps you meant the current version, 2.5.1? > What I'm wondering is why: > > 1) It is so consistent - the 'stop' happens at nearly the exact same point > in every test run, and for nearly the exact same duration - and it happens > multiple times during the test run, in the same pattern. > > 2) I can increase the load 10X (even more if I want), but without the > 'Retrieve All Embedded...' enabled, and it doesn’t happen. > > If it were resource related on the JMeter box, I would guess that changing > the JVM memory config would show significant changes in behavior (like it > would take longer to encounter the first 'stop'). If it were resource > related on the servers, I would expect to see some sort of contention on the > servers. > > Tomorrow I will try the thread dump stuff - to see if there is anything > obvious in that output. > > From an "external" view - it looks like JMeter has some sort of 'pause' > built into it. Since all machines involved are nearly completely 'idle' when > the stop happens - it seems unlikely that I've run out of resources. The > other thought is that there is a deadlock somewhere - and something has to > timeout before it can free up the deadlock and start again. But if that were > the case, I would expect it to happen at slightly different intervals in > each test run - and not repeat in the same consistent pattern during the > test run. > > Some thoughts on what I think I've ruled out: > > 1) If it were the network 'backing off' I would expect the PerfMon listener > to stop updating during the 'stop' - but it trundles happily along. > 2) If it were JVM garbage collections - I would have expected to see them > running (or at least a 'stop-the-world' gc right before/after the 'stop' > period) during the 'stop' period - but there are no gc events at all during > the 'stop' period. > 3) If it were WinXP resources, I would have expected the PerfMon listener to > show some activity during the 'stop' period on the WinXP (JMeter client) box > - nothing like that appears. > > I'll look at the thread dumps tomorrow and get back to you guys. Thanks for > helping me ponder this. > > -- > Robin D. Wilson > Sr. Director of Web Development > KingsIsle Entertainment, Inc. > VOICE: 512-777-1861 > www.KingsIsle.com > > > -----Original Message----- > From: sebb [mailto:[email protected]] > Sent: Wednesday, December 14, 2011 5:27 PM > To: JMeter Users List > Subject: Re: Strange 'pause' activity during testing > > On 14 December 2011 22:20, Deepak Shetty <[email protected]> wrote: >> I think the original post mentioned that GC was not running. >> >> can you take a thread dump when there is a pause and see what the >> threads are waiting on? >> > > I would also try running JMeter in non-GUI mode; GUI mode is very expensive > in resources. > > Ensure all non-essential Listeners are disabled, see: > > http://jmeter.apache.org/usermanual/best-practices.html#lean_mean > >> >> On Wed, Dec 14, 2011 at 1:56 PM, Kirk <[email protected]> wrote: >> >>> might be GC.. me be that JMeter's threads are all hung up in your >>> server doing stuff. >>> >>> Regards, >>> Kirk >>> On Dec 14, 2011, at 10:35 PM, Robin D. Wilson wrote: >>> >>> > I have a marginally complicated test case that performs a > 'registration' >>> on >>> > my site. It gets the home page, POSTS a form, gets the response, >>> > POSTS another form, gets that response, and then completes. The >>> > test runs fine with 100 threads, and 30000 iterations - IF I don't >>> > "Retrieve All >>> Embedded >>> > Resources from HTML Files". In this mode, I am really testing the >>> throughput >>> > of my 'tomcat' application, not the other elements of my system. >>> > (I'm assuming that the other elements are being retrieved from our >>> > Content >>> Data >>> > Network instead of our main system in this case.) >>> > >>> > If I enable the "Retrieve All Embedded Resources from HTML Files" >>> > flag, >>> and >>> > tune the test down to 10 threads with 3000 total iterations, I >>> > notice a >>> very >>> > strange behavior. The test runs along at a pretty good clip for the >>> > first >>> > ~600 iterations (about 1 min, 25 seconds into the run), and then it >>> > just stops making requests for about 35 seconds. Then, it picks >>> > back up again >>> and >>> > runs for another 1 m 25s, and then stops again for 35 seconds... (NOTE: >>> with >>> > the 10 threads, 3000 total iterations - but with "Retrieve All >>> Embedded..." >>> > disabled - I don't see the 'stop' behavior either - so it isn't >>> > caused by tuning it down...) >>> > >>> > I recently added the "Perfmon Metrics Collector" to the test >>> > script, so I could see if one of the servers was maxed out - but it >>> > looks like all the servers are idle during the 'stop' period. >>> > Likewise, I added the Perfmon >>> for >>> > the "localhost" (running the JMeter test) to see if it was swamped >>> > - but >>> it >>> > too is idle during the 'stop'. I swapped out our network switch >>> > (the test environment is on an isolated network switch) with a >>> > _much_ higher >>> capacity >>> > switch - in case there was a network issue, still no change. >>> > >>> > I'm running out of ideas for things to check - so I thought I'd ask >>> > you >>> guys >>> > if you have any suggestions of things I should look at. >>> > >>> > My system consists of: >>> > >>> > WinXP - running JMeter 2.4.1 - driving the test script in GUI >>> > mode >>> > Server 1 - running Red Hat Linux, with "Apache (2.2.21)" as >>> > the web server - using AJP Proxy to Server 2 >>> > Server 2 - running Red Hat Linux, with Tomcat 7.0.21 as the >>> > App Server - connecting through Hibernate to Server 3 >>> > Server 3 - running Red Hat Linux with MySQL 5.x as the DB >>> > Server >>> > >>> > All 4 machines are running on a private switched network (32Gbs >>> backplane). >>> > >>> > The requests are downloading about 3MB total (per thread per >>> > iteration) >>> over >>> > 4 main URL requests, and 30+ 'Retrieve All Embedded' requests. >>> > >>> > At first I thought it was the network - but the new switch seemed >>> > to deny that thought (the old switch had a much slower backplane). >>> > Also, I'm >>> having >>> > no trouble collecting the PerfMon data during the 'stop' period - >>> > so the network is still functioning just fine... >>> > Then I thought it might be garbage collection on the tomcat - but I >>> watched >>> > the gc.log - and it doesn't do any GCs during the 'stop' period. >>> > Then I thought it might be the garbage collection on the JMeter >>> > side, so >>> I >>> > started the JMeter.bat from a 'cmd' prompt with gc logging enabled >>> > - it doesn't do any GCs during the 'stop' period either. >>> > >>> > The apache, tomcat, and DB are all 'idle' (no CPU to speak of, no >>> > network I/O, no disk I/O, etc.) during the 'stop' period. >>> > >>> > The JMeter box (WinXP) is doing very little during that time too (I >>> > attribute the little bit of activity to displaying the PerfMon >>> > graphs, >>> and >>> > Remote Desktop display to my desktop computer)... >>> > >>> > -- >>> > Robin D. Wilson >>> > Sr. Director of Web Development >>> > KingsIsle Entertainment, Inc. >>> > VOICE: 512-777-1861 >>> > www.KingsIsle.com >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------- >>> > -- To unsubscribe, e-mail: [email protected] >>> > For additional commands, e-mail: [email protected] >>> > >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
