anilgursel opened a new pull request #371: URL: https://github.com/apache/tomcat/pull/371
…nections Under regular load, we observe NIO2 has better CPU utilization and slightly better latency compared to NIO1. However, under heavy load or during spikes, we observed that the 99th percentile latency of NIO2 is significantly worse compared to NIO1. - We see that the latency is worse compared to NIO1 only for the requests that end up establishing a new connection. - When `maxKeepAliveRequests=-1`, we do not observe the latency degradation with NIO2 at all. But, we need and use Tomcat’s `maxKeepAliveRequests` feature, so setting it to `-1` is not an option. - The problem occurs only during spikes/high load, e.g., maxThreads=40, while the concurrent users sending requests is 100. - The issue happens only for blocking scenarios, where Tomcat threads are blocked during request processing. When non-blocking programming is used (where Tomcat threads are freed quickly), we do not see any degradation compared to NIO1. However, we have many applications that would not be able to adopt non-blocking anytime soon. While looking at Tomcat code, we saw that the `accept` is scheduled to be run on the thread pool. Our hypothesis was that this scheduling was causing the `accept` to be delayed significantly under heavy load. So, for a request with new connection establishment, it needs to go through the `TaskQueue` twice. So, just for testing purpose we set `maxConnections=-1` to see the impact of immediate `accept` call (and in our test the number of concurrent connections does not get close to `maxConnections`), we observed that the 99th percentile latency problem with NIO2 disappeared. Accordingly, proposing a change to call `accept` immediately if the number of connections are not close to `maxConnections`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
