I have done quite a work with both tools. I used to often get requests to test
the use case with JMeter whenever i gave bad LR timings to the project. Then i
use to repeat the test with Jmeter and i used to get difference of around 2~5%
variation.
So i believed JMeter is developer's tool and LR is QA's tool that's real reason
for 5% variation though they do same things just that they are implemented
differently.
Please see generally blaming the toolset is not a good approach to take.
n fact, as I said, the second, third, fourth, etc. iterations of JMeter's
script give much lower times than the first one. So there is some kind of
overhead/time lost in the first iteration
This is always and almost due to caching and its good thing to do.
I suggest you revisit your test and see if you might be overlooking. Ask
yourself these questions ?
How many requests do you make in each iterations ? how many of them are static
? how many of them involves caching ? What is size of each request ? Are they
http or https calls ? if yes how many of them are http or https calls ? how
many external systems does your request touch ? .. i can think of many such
general questions..
On Thursday, May 27, 2021, 11:42:42 AM EDT, Felix Schumacher
<felix.schumac...@internetallee.de> wrote:
Am 26.05.21 um 21:55 schrieb Gabriele D'Antona:
> Hello Felix,
> I will share the script if we really need it later.
>
> I'm using just one thread and one or more iterations.
>
> This is the result with 3 iterations:
WIth one thread?
Still, we can only guess, what is happening here, as we don't know what
kind of elements you are using.
Assuming you use http samplers, only, than all requests above 100 ms
will probably be the problem of the server and not the client (assuming
further, that you have a good connection between them).
In my eyes the 1.2 seconds for the initial connection are way too long,
When I test a http sampler against googles main page, I get a "Connect"
values of 40-80.
If you want to help us help you, you have to give more information about
the test plan. Sorry.
Apart from that, always remember, that
* Java will perform best after a warm up period (that means, you should
let your tests run long enough to get over the initial warm up phase)
* Java has a Garbage Collector, which can introduce spikes in your
measured response time (remove those outliers or be aware of them)
* It might not be the problem of the client (at least think about it :))
* Different elements of JMeter have different performance implications
(XPath might be more expensive than a plain text comparison)
* If you are familiar with Java, you can try to look deeper into your
suspected problems using tools like jstack, visualvm, jmc or any other
profiler.
Felix
>
> timeStamp,elapsed,label,responseCode,responseMessage,success,failureMessage,Latency,SampleCount,ErrorCount,Hostname,Connect
> 2021/05/26 21:49:58.014,1789,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,1247
> 2021/05/26 21:50:00.012,8838,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:08.888,201,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.094,297,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:09.393,784,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:10.181,4132,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.331,429,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:14.767,6270,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.051,182,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:21.235,1353,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:22.593,619,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:23.215,3483,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:26.710,438,1_MFP_Homepage,200,"Number of samples in
> transaction : 9, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:27.152,6397,2_MFP_Login,200,"Number of samples in
> transaction : 13, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.558,194,3_MFP_InfoOfferta,200,"Number of samples in
> transaction : 1, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:33.754,281,4_MFP_InfoConto,200,"Number of samples in
> transaction : 2, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.038,549,5_MFP_InfoConsumi,200,"Number of samples in
> transaction : 3, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
> 2021/05/26 21:50:34.590,3719,6_MFP_Logout,200,"Number of samples in
> transaction : 16, number of failing samples : 0",true,,0,1,0,SPXMIW3825,0
>
> As you can see, the 2_MFP_Login transaction starts with a time of 8.8
> seconds, then goes down to 6.2-6.3 in the second and third iteration.
> I tried to switch the implementation from "Java" to "HttpClient4", maybe
> this helped a bit.
>
> Thanks
>
>
>
> Il giorno mer 26 mag 2021 alle ore 21:37 Felix Schumacher <
> felix.schumac...@internetallee.de> ha scritto:
>
>> Am 26.05.21 um 21:17 schrieb Gabriele D'Antona:
>>
>> Hello Felix,
>> thanks for your answer.
>>
>> Unfortunately, I can't share the script because it contains private data
>> like passwords and URLs.
>>
>> Maybe you can obscure/anonymize/remove those?
>>
>>
>> My java version is 11.0.3 2019-04-16 LTS, the OS is Windows 10.
>>
>> Which version of JMeter? How may threads are you using, how many requests
>> per second are you trying to reach, how much network traffic do you expect?
>>
>>
>> I can share the jtl file with one iteration of the test. As you can see,
>> the second transaction (2_MFP_Login) takes almost 8 seconds, while on the
>> BSM it takes something like 3,5 seconds.
>> As I said, if I increment the number of iterations, I get better times on
>> the subsequent runs (6.5 seconds for that transaction), but still far from
>> BSM.
>>
>> The list tends to strip of attachements. It seems to have removed yours,
>> too.
>>
>> Can you describe the test elements, that you are using? What is inside the
>> transaction that takes eight seconds?
>>
>> Felix
>>
>> Thanks
>>
>>
>> Il giorno mer 26 mag 2021 alle ore 21:10 Felix Schumacher <
>> felix.schumac...@internetallee.de> ha scritto:
>>
>>> Am 26.05.21 um 21:00 schrieb Gabriele D'Antona:
>>>> Hello,
>>>> I'm a consultant working for a customer that wants to test JMeter as an
>>>> alternative to HP BSM (now owned by Microfocus).
>>>>
>>>> I registered one navigation that is (or at least shoudl be) identical to
>>>> the BSM one (same URLs, same methods, same parameters). When comparing
>>> the
>>>> results, even on single transactions, the times registered by JMeter
>>> are 2x
>>>> the BSM ones. I'm talking of simple HTTPs calls, nothing fancy. I'm also
>>>> launching the jmx project from the command line (no GUI).
>>>>
>>>> I noticed that, if I run the jmx project with 3 or 4 iterations, the
>>> first
>>>> iteration is 2x longer than the BSM one, the subsequent iterations are
>>>> faster, but still 1.5x if compared to BSM.
>>>>
>>>> I hope you can give me some suggestions on this problem.
>>>>
>>>> Also, is there a way to fill the resulting jtl file only with the last
>>>> transaction? This would help having lower response times.
>>> You are quite vague on how you are setting up the plan and what versions
>>> of Java, JMeter or OS you are using.
>>>
>>> Can you be more specific? Maybe give a real example?
>>>
>>> Felix
>>>
>>>> Many thanks.
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org
>> For additional commands, e-mail: user-h...@jmeter.apache.org
>>
>>
