On 11/04/2021 11:03, Peter Chamberlain wrote:
<snip/>
I've been investigating this some more, as I'm not convinced nio2 isn't
behaving strangely in this case. I think there may of been some sort of
reversion as it is much less likely to refuse connections for nio2 in
tomcat 9.0.13 when compared to 9.0.14. I'm wondering if it has something to
do with:
Avoid using a dedicated thread for accept on the NIO2 connector,
it is always less efficient. (remm)
And if it is hitting some sort of accept thread starvation case when it is
fully loaded. In tomcat 9.0.13 I can hit a maxTheads=200 nio2 connector
with 5000 jmeter threads and not experience a connection refused, but in
9.0.14 I can't reach 1000 without refused connections. It doesn't seem to
be related to forwards or redirects either. If I just sleep for 1500
milliseconds for every servlet run and not redirect or forward and it
behaves the same.
We've been using nio2 in our tomcats exclusively for some time, as we hit
an issue with nio in the past (can't remember what it was, it is likely
fixed by now I would think), so I guess we're more likely to notice this
sort of thing.
I think you are asking the wrong question(s). 200 threads with a 1500ms
wait means I would expect Tomcat to be processing ~133 requests per
second. (Assuming you have at least 200 client threads as well). Higher
numbers of client threads, the timeouts configured on the client, the
timeouts configured on Tomcat, the accept count etc shouldn't change the
requests per second results. What will change is the failure scenarios
you observe - and I think that is what you are seeing here between
9.0.13 and 9.0.14. 9.0.13 might be accepting more connections but that
doesn't mean those connections are being processed faster. Depending on
timeouts, they might (eventually) get processed or they might timeout.
You might want to try the following:
- Limit the number of loops to, say, 10 so you get 50,000 requests. Look
at the response time stats. What is the average? What is the min/max?
- Repeat the test. Do the results remain consistent?
- Repeat the test with more loops. Do the results remain consistent?
- Repeat the test with fewer client threads. At what point do you start
to get consistent results?
It may well be that changes to Tomcat over time have changed the way
Tomcat behaves under various (overloaded network) failure scenarios.
My reading of the change that you reference above does mean that Tomcat
will only accept a new connection over NIO2 when it has a processing
thread available to process it. That will change the way Tomcat behaves
when presented with a large spike of new connections. (Significantly)
increasing the acceptCount (a.k.a. backlog) to more than the number
connections expected in a single "spike" in 9.0.14 should give 9.0.13
like behaviour.
HTH,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
