Hi,
Few more observations:
- With r1715087:
- CPU is between 15 and 18%
- Network traffic : 1276 Mbits/s IN and 96 Mbits/s OUT
- Thread dump show 100 threads of 1024 reading on socketRead
- With 3.0.0:
- CPU is between 7 and 10%
- Network traffic : 528 Mbits/s IN and 56 Mbits/s OUT
- Thread dump show 600 threads of 1024 reading on socketRead
So it appears connect timeout is just a side effect of a big slowdown in
socket reading with HTTPClient or an issue in JMeter which I don't see.
Regards
On Sun, Jan 31, 2016 at 2:58 PM, Philippe Mouawad <
[email protected]> wrote:
> Hello,
> I added the debug and noticed a bug in connect time computing:
>
> - https://bz.apache.org/bugzilla/show_bug.cgi?id=58947
>
> Which I fixed.
>
> Once fixed connectTime is equal to something like 10003 to 10010 ms (I
> have configured it to 10000 as a timeout)
> So looks regular here.
>
> So for now still no definite idea of what the issue could be
> Regards
>
>
>
> On Sat, Jan 30, 2016 at 1:48 PM, Philippe Mouawad <
> [email protected]> wrote:
>
>> Hi sebb,
>> Do you have some idea in mind ? What information do you want ?
>>
>> For info, I made a test with same "working" jmeter version rr1715087 and
>> only changed httpclient/core/mime to 4.5.2/4.4.4/4.5.1 and I reproduce the
>> issue.
>>
>> Regards
>>
>> On Sat, Jan 30, 2016 at 1:40 AM, sebb <[email protected]> wrote:
>>
>>> On 30 January 2016 at 00:13, Philippe Mouawad
>>> <[email protected]> wrote:
>>> > Well,
>>> > it's a Connect Timeout (connect Timeout being set to 10 s), I cannot
>>> tell
>>> > as Timeout occurs on an embedded resources.
>>> > But it seems issues is related to Embedded resources download.
>>>
>>> So add some debug then to show the actual times.
>>>
>>> > Regards
>>> >
>>> > On Sat, Jan 30, 2016 at 1:00 AM, sebb <[email protected]> wrote:
>>> >
>>> >> On 29 January 2016 at 23:47, Philippe Mouawad
>>> >> <[email protected]> wrote:
>>> >> > On Saturday, January 30, 2016, sebb <[email protected]> wrote:
>>> >> >
>>> >> >> On 29 January 2016 at 23:31, Philippe Mouawad
>>> >> >> <[email protected] <javascript:;>> wrote:
>>> >> >> > Hello,
>>> >> >> > I made a real load test today using nightly build and faced an
>>> >> important
>>> >> >> > issue.
>>> >> >> >
>>> >> >> > Here are the details:
>>> >> >> > - Test uses 1000 Threads on 1 Instance
>>> >> >> > - It uses "Download Embedded Resources"
>>> >> >> > - Socket Timeout is set to 10s
>>> >> >> >
>>> >> >> > There is no overloading of the machine, no impacting GC
>>> >> >> >
>>> >> >> > Very rapidly, I start getting a lot of errors:
>>> >> >> >
>>> >> >> >
>>> >> >> > - Non HTTP response code:
>>> >> org.apache.http.conn.ConnectTimeoutException
>>> >> >> > message:Non HTTP response message: Connect to www.foo.com:80
>>> timed
>>> >> >> out
>>> >> >>
>>> >> >> What is the elapsed time till the timeout occurs?
>>> >> >
>>> >> >
>>> >> > In the initial test, it happens after 2 minutes at around 300.
>>> threads.
>>> >> >
>>> >> > with a test at 300 threads and a very rapid rampup (5 seconds, with
>>> a
>>> >> > startup delay of 10 seconds) to reproduce it happens after 16
>>> seconds
>>> >> >
>>> >>
>>> >> I meant - how long between the start of a sample until the timeout?
>>> >> Does it vary much between samples?
>>> >>
>>> >> >>
>>> >> >> >
>>> >> >> > Rate of error varies between 15% and 30%.
>>> >> >> > Note that if I navigate on the application, I don't face the
>>> errors.
>>> >> >> >
>>> >> >> > I ran the same test using exactly the same configuration:
>>> >> >> >
>>> >> >> > - Same machine
>>> >> >> > - Same JVM version and tuning
>>> >> >> > - Same user.properties
>>> >> >> > - Same hc.parameters
>>> >> >> >
>>> >> >> > But jmeter r1715087
>>> >> >> > And error rate is 0.30%.
>>> >> >> >
>>> >> >> >
>>> >> >> > Note the target server has a load balancer that returns a
>>> keep-alive
>>> >> >> > duration set to 2 (2 seconds).
>>> >> >> >
>>> >> >> > This issue is a blocker one for the release of next version.
>>> >> >> > I compared code with revision 1715087 and I don't see many
>>> changes in
>>> >> >> > HTTPHC4Impl that would explain this regression.
>>> >> >> > I commented out some suspects , retried but I get same results.
>>> >> >> >
>>> >> >> >
>>> >> >> > I also upgraded to HttpClient 4.5.2 and uncommented the code
>>> expected
>>> >> to
>>> >> >> be
>>> >> >> > added, same results.
>>> >> >> >
>>> >> >> > So for now I tend to suspect an issue in HttpClient/Core.
>>> >> >> > --
>>> >> >> > Regards
>>> >> >> > Philippe M.
>>> >> >>
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Cordialement.
>>> >> > Philippe Mouawad.
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Cordialement.
>>> > Philippe Mouawad.
>>>
>>
>>
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
