I don't think that's what I'm seeing. The test does 600+ iterations (sorry, I miscommunicated earlier - not 'samples', but iterations) - that's ~2400 samples with each requisite embedded resource set per sample - so ~24000 GET and POST requests are fulfilled before the first pause.
Also, if it were my application pausing, I would expect to see similar behavior when I run without the embedded resources (all of the embedded resources are being served as static files directly from the Apache server, they never even hit the tomcat). Especially considering I can increase the thread count 10X and still not see an application pause. Without the embedded files, each sampler downloads 50-70KB. With the embedded files, each sampler downloads 500-900KB. So there is definitely a _lot_ more data (more than 10X) being transferred. HOWEVER, when the 'pause' occurs - the network is idle on all machines in the environment - so if its busy downloading stuff, it's doing it without any bandwidth being used. -- Robin D. Wilson Sr. Director of Web Development KingsIsle Entertainment, Inc. VOICE: 512-777-1861 www.KingsIsle.com -----Original Message----- From: Deepak Goel [mailto:deic...@gmail.com] Sent: Thursday, December 15, 2011 10:35 AM To: JMeter Users List Subject: Re: Strange 'pause' activity during testing Hey It looks like common sense... The embedded files in the html page are about 6 odd per original page. So if you have 10 threads firing the load, Jmeter has to make 60 odd connections to the web server to complete these transactions. The embedded download from the web server to Jmeter is taking about 300ms. So your test makes a pause of 35 odd seconds and then continues because it has to download all the embedded stuff and then refire the next 10 transactions. The test hardly pauses, it is completing the embedded download.... :) Deepak On 12/15/11, Adrian Speteanu <asp.ad...@gmail.com> wrote: > I agree that jmeter rarely performs poorly in non-GUI mode, unless you > allocate to much memory to it. > > On Thu, Dec 15, 2011 at 1:12 PM, Adrian Speteanu <asp.ad...@gmail.com>wrote: > >> In my experience, "stange pauses" usually points to the application. >> >> Of course, you have to make sure its not you (overlook in the test >> script, jmeter's configuration), or the test client environment (OS, >> network). BUT after all that is exhausted is usually the application >> or one of the layers of the system under test (like the database, >> usually more trivial stuff like a small connection pool to the >> database, or GC of the application under test). >> >> If you suspect its GC, divide the load you generate with the script >> in 3, and instead of testing with one jmeter instance, use 3 (if you >> can test like this on 3 different machines - the better). Also >> allocate less memory to jmeter now. What's the probability that all >> instances get paused by GC at the same time? Also, if you allocate >> less memory, stop-the-world GC takes less time. If you still have >> response times pauses in the same intervals - it has to be the system under test. Note: >> >> On Thu, Dec 15, 2011 at 3:51 AM, Robin D. Wilson >> <rwils...@gmail.com>wrote: >> >>> Yes it is. I can turn it off if you think that matters. >>> >>> -- >>> Robin D. Wilson >>> >>> >>> >>> On Dec 14, 2011, at 6:42 PM, ba...@baassiri.ca wrote: >>> >>> > I'm curious to know if the system under test is https >>> > >>> > >>> > -----Original Message----- >>> > From: sebb <seb...@gmail.com> >>> > Date: Thu, 15 Dec 2011 00:35:39 >>> > To: JMeter Users List<user@jmeter.apache.org> >>> > Reply-To: "JMeter Users List" <user@jmeter.apache.org> >>> > Subject: Re: Strange 'pause' activity during testing >>> > >>> > On 15 December 2011 00:22, Robin D. Wilson <rwils...@gmail.com> wrote: >>> >> I can use non-GUI mode for testing out the problem, but with the >>> >> exact >>> same >>> >> test - just switching the 'Retrieve All Embedded...' disabled, >>> >> the >>> problem >>> >> goes away. >>> >> >>> >> The only listeners I have enabled are the summary listener - the >>> PerfMon >>> >> listeners (1 for each server) - and a 'tree' listener, but it >>> >> only >>> reports >>> >> 'errors'. >>> >> >>> >> I can run the test with 'Retrieve All Embedded...' enabled, I see >>> >> the problem every time. >>> > >>> > Maybe then there is a problem with the scanning of the HTML to >>> > extract the embedded resources, or maybe one of the embedded >>> > resources is a tar-pit. >>> > >>> > Do all the expected embedded resources get downloaded? >>> > >>> > Are there any unusually long elapsed times for embedded resources? >>> > >>> > Or large gaps between the parent sample download completion and >>> > the start of the first embedded resource? >>> > >>> > That would suggest the page took a while to parse. >>> > >>> > You'll need to select the optiion to "save subresults" in order to >>> > see the embedded samples. >>> > >>> >> If I just uncheck the box in the Request Defaults and re-run the >>> >> test, >>> it >>> >> never happens... So are you suggesting that I've hit some >>> >> arbitrary >>> limit of >>> >> the GUI mode? >>> > >>> > No, just that GUI mode is inherently more resource intensive. >>> > >>> > BTW, you wrote you were running JMeter 2.4.1 - that does not >>> > exist, so perhaps you meant the current version, 2.5.1? >>> > >>> > >>> >> What I'm wondering is why: >>> >> >>> >> 1) It is so consistent - the 'stop' happens at nearly the exact >>> >> same >>> point >>> >> in every test run, and for nearly the exact same duration - and >>> >> it >>> happens >>> >> multiple times during the test run, in the same pattern. >>> >> >>> >> 2) I can increase the load 10X (even more if I want), but without >>> >> the 'Retrieve All Embedded...' enabled, and it doesn't happen. >>> >> >>> >> If it were resource related on the JMeter box, I would guess that >>> changing >>> >> the JVM memory config would show significant changes in behavior >>> >> (like >>> it >>> >> would take longer to encounter the first 'stop'). If it were >>> >> resource related on the servers, I would expect to see some sort >>> >> of contention >>> on the >>> >> servers. >>> >> >>> >> Tomorrow I will try the thread dump stuff - to see if there is >>> >> anything obvious in that output. >>> >> >>> >> From an "external" view - it looks like JMeter has some sort of >>> >> 'pause' >>> >> built into it. Since all machines involved are nearly completely >>> 'idle' when >>> >> the stop happens - it seems unlikely that I've run out of resources. >>> The >>> >> other thought is that there is a deadlock somewhere - and >>> >> something >>> has to >>> >> timeout before it can free up the deadlock and start again. But >>> >> if >>> that were >>> >> the case, I would expect it to happen at slightly different >>> >> intervals >>> in >>> >> each test run - and not repeat in the same consistent pattern >>> >> during >>> the >>> >> test run. >>> >> >>> >> Some thoughts on what I think I've ruled out: >>> >> >>> >> 1) If it were the network 'backing off' I would expect the >>> >> PerfMon >>> listener >>> >> to stop updating during the 'stop' - but it trundles happily along. >>> >> 2) If it were JVM garbage collections - I would have expected to >>> >> see >>> them >>> >> running (or at least a 'stop-the-world' gc right before/after the >>> 'stop' >>> >> period) during the 'stop' period - but there are no gc events at >>> >> all >>> during >>> >> the 'stop' period. >>> >> 3) If it were WinXP resources, I would have expected the PerfMon >>> listener to >>> >> show some activity during the 'stop' period on the WinXP (JMeter >>> client) box >>> >> - nothing like that appears. >>> >> >>> >> I'll look at the thread dumps tomorrow and get back to you guys. >>> Thanks for >>> >> helping me ponder this. >>> >> >>> >> -- >>> >> Robin D. Wilson >>> >> Sr. Director of Web Development >>> >> KingsIsle Entertainment, Inc. >>> >> VOICE: 512-777-1861 >>> >> www.KingsIsle.com >>> >> >>> >> >>> >> -----Original Message----- >>> >> From: sebb [mailto:seb...@gmail.com] >>> >> Sent: Wednesday, December 14, 2011 5:27 PM >>> >> To: JMeter Users List >>> >> Subject: Re: Strange 'pause' activity during testing >>> >> >>> >> On 14 December 2011 22:20, Deepak Shetty <shet...@gmail.com> wrote: >>> >>> I think the original post mentioned that GC was not running. >>> >>> >>> >>> can you take a thread dump when there is a pause and see what >>> >>> the threads are waiting on? >>> >>> >>> >> >>> >> I would also try running JMeter in non-GUI mode; GUI mode is very >>> expensive >>> >> in resources. >>> >> >>> >> Ensure all non-essential Listeners are disabled, see: >>> >> >>> >> http://jmeter.apache.org/usermanual/best-practices.html#lean_mean >>> >> >>> >>> >>> >>> On Wed, Dec 14, 2011 at 1:56 PM, Kirk >>> >>> <kirk.pepperd...@gmail.com> >>> wrote: >>> >>> >>> >>>> might be GC.. me be that JMeter's threads are all hung up in >>> >>>> your server doing stuff. >>> >>>> >>> >>>> Regards, >>> >>>> Kirk >>> >>>> On Dec 14, 2011, at 10:35 PM, Robin D. Wilson wrote: >>> >>>> >>> >>>>> I have a marginally complicated test case that performs a >>> >> 'registration' >>> >>>> on >>> >>>>> my site. It gets the home page, POSTS a form, gets the >>> >>>>> response, POSTS another form, gets that response, and then >>> >>>>> completes. The test runs fine with 100 threads, and 30000 >>> >>>>> iterations - IF I don't "Retrieve All >>> >>>> Embedded >>> >>>>> Resources from HTML Files". In this mode, I am really testing >>> >>>>> the >>> >>>> throughput >>> >>>>> of my 'tomcat' application, not the other elements of my system. >>> >>>>> (I'm assuming that the other elements are being retrieved from >>> >>>>> our Content >>> >>>> Data >>> >>>>> Network instead of our main system in this case.) >>> >>>>> >>> >>>>> If I enable the "Retrieve All Embedded Resources from HTML Files" >>> >>>>> flag, >>> >>>> and >>> >>>>> tune the test down to 10 threads with 3000 total iterations, I >>> >>>>> notice a >>> >>>> very >>> >>>>> strange behavior. The test runs along at a pretty good clip >>> >>>>> for the first >>> >>>>> ~600 iterations (about 1 min, 25 seconds into the run), and >>> >>>>> then it just stops making requests for about 35 seconds. Then, >>> >>>>> it picks back up again >>> >>>> and >>> >>>>> runs for another 1 m 25s, and then stops again for 35 seconds... >>> (NOTE: >>> >>>> with >>> >>>>> the 10 threads, 3000 total iterations - but with "Retrieve All >>> >>>> Embedded..." >>> >>>>> disabled - I don't see the 'stop' behavior either - so it >>> >>>>> isn't caused by tuning it down...) >>> >>>>> >>> >>>>> I recently added the "Perfmon Metrics Collector" to the test >>> >>>>> script, so I could see if one of the servers was maxed out - >>> >>>>> but it looks like all the servers are idle during the 'stop' period. >>> >>>>> Likewise, I added the Perfmon >>> >>>> for >>> >>>>> the "localhost" (running the JMeter test) to see if it was >>> >>>>> swamped >>> >>>>> - but >>> >>>> it >>> >>>>> too is idle during the 'stop'. I swapped out our network >>> >>>>> switch (the test environment is on an isolated network switch) >>> >>>>> with a _much_ higher >>> >>>> capacity >>> >>>>> switch - in case there was a network issue, still no change. >>> >>>>> >>> >>>>> I'm running out of ideas for things to check - so I thought >>> >>>>> I'd ask you >>> >>>> guys >>> >>>>> if you have any suggestions of things I should look at. >>> >>>>> >>> >>>>> My system consists of: >>> >>>>> >>> >>>>> WinXP - running JMeter 2.4.1 - driving the test script >>> >>>>> in GUI mode >>> >>>>> Server 1 - running Red Hat Linux, with "Apache (2.2.21)" >>> >>>>> as the web server - using AJP Proxy to Server 2 >>> >>>>> Server 2 - running Red Hat Linux, with Tomcat 7.0.21 as >>> >>>>> the App Server - connecting through Hibernate to Server 3 >>> >>>>> Server 3 - running Red Hat Linux with MySQL 5.x as the >>> >>>>> DB Server >>> >>>>> >>> >>>>> All 4 machines are running on a private switched network >>> >>>>> (32Gbs >>> >>>> backplane). >>> >>>>> >>> >>>>> The requests are downloading about 3MB total (per thread per >>> >>>>> iteration) >>> >>>> over >>> >>>>> 4 main URL requests, and 30+ 'Retrieve All Embedded' requests. >>> >>>>> >>> >>>>> At first I thought it was the network - but the new switch >>> >>>>> seemed to deny that thought (the old switch had a much slower backplane). >>> >>>>> Also, I'm >>> >>>> having >>> >>>>> no trouble collecting the PerfMon data during the 'stop' >>> >>>>> period - so the network is still functioning just fine... >>> >>>>> Then I thought it might be garbage collection on the tomcat - >>> >>>>> but I >>> >>>> watched >>> >>>>> the gc.log - and it doesn't do any GCs during the 'stop' period. >>> >>>>> Then I thought it might be the garbage collection on the >>> >>>>> JMeter side, so >>> >>>> I >>> >>>>> started the JMeter.bat from a 'cmd' prompt with gc logging >>> >>>>> enabled >>> >>>>> - it doesn't do any GCs during the 'stop' period either. >>> >>>>> >>> >>>>> The apache, tomcat, and DB are all 'idle' (no CPU to speak of, >>> >>>>> no network I/O, no disk I/O, etc.) during the 'stop' period. >>> >>>>> >>> >>>>> The JMeter box (WinXP) is doing very little during that time >>> >>>>> too (I attribute the little bit of activity to displaying the >>> >>>>> PerfMon graphs, >>> >>>> and >>> >>>>> Remote Desktop display to my desktop computer)... >>> >>>>> >>> >>>>> -- >>> >>>>> Robin D. Wilson >>> >>>>> Sr. Director of Web Development KingsIsle Entertainment, Inc. >>> >>>>> VOICE: 512-777-1861 >>> >>>>> www.KingsIsle.com >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> ------------------------------------------------------------------- >>> >>>>> -- To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> >>>>> For additional commands, e-mail: user-h...@jmeter.apache.org >>> >>>>> >>> >>>> >>> >>>> >>> >>>> --------------------------------------------------------------------- >>> >>>> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> >>>> For additional commands, e-mail: user-h...@jmeter.apache.org >>> >>>> >>> >>>> >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> >> For additional commands, e-mail: user-h...@jmeter.apache.org >>> >> >>> >> >>> >> --------------------------------------------------------------------- >>> >> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> >> For additional commands, e-mail: user-h...@jmeter.apache.org >>> >> >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> > For additional commands, e-mail: user-h...@jmeter.apache.org >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org >>> For additional commands, e-mail: user-h...@jmeter.apache.org >>> >>> >> > -- Namaskara~Nalama~Guten Tag~Bonjour -- Keigu Deepak +91-9765089593 deic...@gmail.com http://www.simtree.net Skype: thumsupdeicool Google talk: deicool Blog: http://loveandfearless.wordpress.com Facebook: http://www.facebook.com/deicool "Contribute to the world, environment and more : http://www.gridrepublic.org " --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
