Thanks sebb for the replies... Here's the deal, I am running the same test script on JM2.4 and JM2.6. I am running in GUI mode. The test script has 3 thread groups - but the first and the last thread group is just a 'timer' I created to log the total elapsed time of the test (the first and last group has 1 thread, and 1 request, and take less than 1 second each to run). The 'real' test is the middle thread group. It has 100 threads (0 ramp), and runs 100 iterations (10,000 total samples). It simply does a 'POST' to a URL, with 15
So the 'elapsed time' I referring to in my test is actually the timestamp taken in the first thread group (in ms since epoch) subtracted from the timestamp taken in the 3rd (last) thread group. That part of my test may only add 2 total seconds to the test, so while it may skew my results slightly - it doesn't explain the vast difference in the 'average' sample duration. According to the Summary Report docs, the "Average" is supposed to be "the average elapsed time of a set of samples". But clearly, if the minimum time it takes to actually get the page is 2 seconds (due to the built-in delay in the cgi-script), there is no way I could have an 'average' elapsed time of less than 2 seconds, yet I'm showing an average elapsed time of ~750 ms... (My "Max" elapsed time shows as only 1198!). When I request the page in Firefox, it takes ~2104ms (using a status bar timer), so I think the cgi script is working correctly.) Sebb asked: >Again, the throughput calculations are based on total test time. Are you sure >the test run times are comparable? The test run times are automatically calculated by the 1st and 3rd thread groups. The ~210 seconds total elapsed time is accurate based on my external measurement too (e.g., it is close to what I can observe with my stopwatch). Both the JM2.4 test and the JM2.6 test are using the exact same ".jmx" test file. >There's clearly something else going on here. I don't believe that the Summary Report is accurately calculating anything except the total number of samples and the Avg. Bytes... The cgi-script I'm using definitely takes 2+ seconds to respond after it gets the request (I've measured this with Firefox directly, and it _never_ gets a response in less than 2 seconds). I even changed the 'sleep' to 9 seconds, and JMeter pauses for that long in recording results (e.g., it shows 100 threads run, then waits 9 seconds, shows another 100 threads, etc.), but the numbers just go up to '1758' Average, and '2415' Max (which is impossible since it is taking 9+ seconds to respond to each request!). It takes over 15 minutes to complete 10,000 samples (and that seems about right - 10000 samples/100 threads * 9 seconds each = 900 seconds). I even went so far as to inject a 2 second sleep in the middle of the response (e.g., pause 2 seconds - send part of the response - pause 2 more seconds - send the rest), I'm still getting average times of ~1000 ms. (That's with 4 seconds of built-in delays, and 2 of those seconds are in the middle of the response.) The browser shows this delay properly, but JMeter isn't calculating it properly. >Please recheck the individual sample response times and see how they compare >to the average. I'm not sure how to do that in JMeter. I can manually hit the page, and it takes about 100ms longer than the built-in delay I have. >If there still appears to be a problem, create a Bugzilla issue and attach: >- JMX test case I'm trying to simplify the test case to the bare minimum case - so the results will be indisputable. I will also include the 'cgi-bin' script that I'm using, so someone else can easily setup the same test. >- log files for JMeter 2.4 and 2.6 Which log files are these? Is it just the 'jmeter.log' that gets created in the 'bin' folder when I run the GUI mode, or do you need another log file? >- CSV result files for 2.4 and 2.6 I can do this. -- Robin D. Wilson Sr. Director of Web Development KingsIsle Entertainment, Inc. VOICE: 512-777-1861 www.KingsIsle.com --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org For additional commands, e-mail: user-h...@jmeter.apache.org
