Re: bindOnInit and maxConnections for AJP connectors
On 21/04/2011 20:21, Mark Thomas wrote: On 06/04/2011 22:51, Tim Whittington wrote: On Wed, Apr 6, 2011 at 11:16 PM, Mark Thomas ma...@apache.org wrote: On 05/04/2011 10:50, Tim Whittington wrote: Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) This is still an issue. I'm still thinking about how to address it. My current thinking is: - BIO: Introduce simulated polling using a short timeout (see below) - NIO: Leave as is - APR: Make maxConnections and pollerSize synonyms - All: Make the default for maxConnections 8192 so it is consistent with the current APR default. The other option is dropping maxConnections entirely from the NIO and APR connectors. That would align the code with the docs. The only downside is that the NIO connector would no longer have an option to limit the connections. I'm not sure that is much of an issue since I don't recall any demands for such a limit from the user community. Apologies for what I expect will turn out to be a long e-mail. I have reached the point where I believe the best way forward is: - remove maxConnections from NIO and APR - remove the ability to set maxConnections for BIO and hard code it to maxThreads - restore the disable keep-alive when 75% BIO threads are in use My reasoning is as follows: - Servlet 3.0 async requests mean that current connections in use may be greater then current threads in use. - This provides potential efficiency savings as less threads are required. - That connections may be greater than threads led to the new maxConnections attribute. - maxConnections maxThreads introduces an issue where a connection with data may be in the connection queue waiting for a thread whilst all the threads are busy doing nothing waiting for data on connections that will eventually time out. - This issue can be fixed with simulated polling. - Testing showed that simulated polling was very CPU intensive (I saw a typical increase from ~40% to ~50% CPU usage with 4 Tomcat threads, 2 'fast' client threads making requests as fast as they could, 10 'slow' client threads making a request every 5s and a pollTime of 100ms on an 8-core machine. - The additional resources required by simulated polling are likely to be greater than those saved by reduced thread usage. - It is therefore better to just increase maxThreads, expecting that not all of them will be used and hard-code maxConnections to the same number as maxThreads. Better still, just use NIO. Further, the complexity of BIO code required to support: - Optional configuration of maxConnections maxThreads - simulated polling when maxConnections maxThreads - auto-disabling of keep-alive for users that don't want the overhead of simulated polling when maxConnections == maxThreads was getting to the point where I had stability concerns. Given the above, and assuming there are no objections, I intend to implement the way forward I set out above tomorrow. Cheers, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 5/3/2011 10:50 AM, Mark Thomas wrote: On 21/04/2011 20:21, Mark Thomas wrote: On 06/04/2011 22:51, Tim Whittington wrote: On Wed, Apr 6, 2011 at 11:16 PM, Mark Thomasma...@apache.org wrote: On 05/04/2011 10:50, Tim Whittington wrote: Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) This is still an issue. I'm still thinking about how to address it. My current thinking is: - BIO: Introduce simulated polling using a short timeout (see below) - NIO: Leave as is - APR: Make maxConnections and pollerSize synonyms - All: Make the default for maxConnections 8192 so it is consistent with the current APR default. The other option is dropping maxConnections entirely from the NIO and APR connectors. That would align the code with the docs. The only downside is that the NIO connector would no longer have an option to limit the connections. I'm not sure that is much of an issue since I don't recall any demands for such a limit from the user community. Apologies for what I expect will turn out to be a long e-mail. I have reached the point where I believe the best way forward is: - remove maxConnections from NIO and APR - remove the ability to set maxConnections for BIO and hard code it to maxThreads -1 maxConnections serves a purpose. It protects against a DoS. There must be a way to configure a system to eventually push back, and not just keep accepting connections. Therefor the maxConnections should stay. - restore the disable keep-alive when75% BIO threads are in use why, make it configurable. I would believe that on many systems, the queued approach that BIO has, still can provide significant improvement in performance. If I don't want queued behavior, I'll just turn off keep alive. My reasoning is as follows: - Servlet 3.0 async requests mean that current connections in use may be greater then current threads in use. - This provides potential efficiency savings as less threads are required. - That connections may be greater than threads led to the new maxConnections attribute. the maxConnections attribute is simply to restrict the acceptor thread from just going on and on. It had nothing to do with servlet 3 or async. It's a protective measure for the server. - maxConnections maxThreads introduces an issue where a connection with data may be in the connection queue waiting for a thread whilst all the threads are busy doing nothing waiting for data on connections that will eventually time out. and this is still better than simply not accepting any connections at all. Right now, in order to work around the blocking aspect, one has to configure acceptCount so that connections are not simply turned away. - This issue can be fixed with simulated polling. I wouldn't do simulated polling. I'm not sure what benefit there would be. - Testing showed that simulated polling was very CPU intensive (I saw a typical increase from ~40% to ~50% CPU usage with 4 Tomcat threads, 2 'fast' client threads making requests as fast as they could, 10 'slow' client threads making a request every 5s and a pollTime of 100ms on an 8-core machine. - The additional resources required by simulated polling are likely to be greater than those saved by reduced thread usage. - It is therefore better to just increase maxThreads, expecting that not all of them will be used and hard-code maxConnections to the same number as maxThreads. Better still, just use NIO. you still need maxConnection Further, the complexity of BIO code required to support: - Optional configuration of maxConnections maxThreads - simulated polling when maxConnections maxThreads - auto-disabling of keep-alive for users that don't want the overhead of simulated polling when maxConnections == maxThreads was getting to the point where I had stability concerns. Given the above, and assuming there are no objections, I intend to implement the way forward I set out above tomorrow. I'm confused about what you are trying to do or achieve. What problems are you trying to solve? This email thread started with two missing attributes. I'd start a new thread describing the problem you are having. best Filip Cheers, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - No virus found in this message. Checked by AVG - www.avg.com Version:
Re: bindOnInit and maxConnections for AJP connectors
On 03/05/2011 18:07, Filip Hanik - Dev Lists wrote: I'm confused about what you are trying to do or achieve. What problems are you trying to solve? This email thread started with two missing attributes. I'd start a new thread describing the problem you are having. OK. I'll pull out the BIO performance issues into a new thread. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 05/04/2011 09:51, Tim Whittington wrote: In the AJP standard implementation docs, the following are not mentioned, although they're properties of AbstractEndpoint and probably should work: - bindOnInit - maxConnections Am I right in assuming these should be possible in the AJP connector (my reading of the code indicates they are - just wanted to check if something arcane was going on)? If fixed the docs for bindOnInit maxConnections is still TBD. Mark If so I'll update the docs. cheers tim - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 06/04/2011 22:51, Tim Whittington wrote: On Wed, Apr 6, 2011 at 11:16 PM, Mark Thomas ma...@apache.org wrote: On 05/04/2011 10:50, Tim Whittington wrote: Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) This is still an issue. I'm still thinking about how to address it. My current thinking is: - BIO: Introduce simulated polling using a short timeout (see below) - NIO: Leave as is - APR: Make maxConnections and pollerSize synonyms - All: Make the default for maxConnections 8192 so it is consistent with the current APR default. The other option is dropping maxConnections entirely from the NIO and APR connectors. That would align the code with the docs. The only downside is that the NIO connector would no longer have an option to limit the connections. I'm not sure that is much of an issue since I don't recall any demands for such a limit from the user community. There are a number of issues with the current BIO implementation. 1. Keep-alive timeouts Fixed. 2. The switch to a queue does result in the possibility of requests with data being delayed by requests without data in keep-alive. Still TODO. 3. HTTP pipe-lining is broken (this is bug 50957 [1]). Fixed. The fix for issue 2 is tricky. The fundamental issue is that to resolve it and to keep maxConnections maxThreads we need NIO like behaviour from a BIO socket which just isn't possible. Fixing 1 will reduce the maximum length of delay that any one request might experience which will help but that won't address the fundamental issue. For sockets in keepalive, I considered trying to fake NIO behaviour by using a read with a timeout of 1ms, catching the SocketTimeoutException and returning them to the back of the queue if there is no data. The overhead of that looks to be around 2-3ms for a 1ms timeout. I'm worried about CPU usage but for a single thread this doesn't seem to be noticeable. More testing with multiple threads is required. The timeout could be tuned by looking at the current number of active threads, size of the queue etc. but it is an ugly hack. Returning to the pre [3] approach of disabling keep-alive once connections 75% of threads would fix this at the price of no longer being able to support maxConnections maxThreads. Yeah, I went down this track as well before getting to the Just use APR/NIO state of mind. It is an ugly hack, but might be workable if the timeout is large enough to stop it being a busy loop on the CPU. With 200 threads, even a 100ms timeout would give you a 'reasonable' throughput. Even if we do this, I still think maxConnections should be somewhat closer to maxThreads than it is now if the BIO connector is being used. We can make the timeout configurable but 100ms shouldn't be too bad. I have a simple, scaled-down test case and this approach looks good in terms of throughput but there are a few kinks I need to iron out. Also the CPU usage looked rather high, although that might be a side-effect of one of the kinks. If the issues can't be ironed out then we'll need a major rethink about this - probably heading back towards a TC6 style solution. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On Fri, Apr 8, 2011 at 2:40 AM, Christopher Schultz ch...@christopherschultz.net wrote: Mark, I understand that a fix has already been applied, but... On 4/6/2011 7:16 AM, Mark Thomas wrote: I thought of two options for issue 3: a) Assign a processor (+ inputbuffer, output buffer etc.) to a socket and don't recycle it until the socket is closed. - Increases memory requirements. - Fixes issue 3 - Retains current request processing order. b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue What about a third option that is essentially (a) except that you trim the input buffer and discard the already-processed request(s) from it. The input buffer stays bound to the request and continues where it left-off when another request processor becomes available. That would maintain scheduling fairness and hopefully not require much in the way of memory usage. Since pipelining requests with entities is not a good idea, the chances of getting a large amount of data in the input buffer is relatively low. There's also a limit on the size of that buffer, too. Would that still represent too large of a memory requirement? The input buffer is 8k by default (max header size), so this could be significant with a large maxConnections. Pruning the buffers to retain only required space would generate a lot of garbage, so probably not a good option either. If the fairness becomes a practical problem, reducing maxKeepAliveRequests (100 by default) would force pipelining clients to the back of the queue regularly. We could go with option a) and use the processor cache size to bound the 'memory vs fairness tradeoff' - e.g. place pipeline connections back on the queue, but add their processor to a 'borrowed' queue without recycling. If the total processor limit is met and the recycled processor cache is exhausted, then a borrowed processor is recalled, the pipelined connection drained, and the processor then made available for use. The default processorCache is 200 (matches maxThreads), so default behaviour would be unfair. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 4/8/2011 2:50 AM, Tim Whittington wrote: The input buffer is 8k by default (max header size), so this could be significant with a large maxConnections. significant in 1990 maybe :) even with 20k connections, this be a drop in the ocean compared to memory usage for today's applications - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
Tim, On 4/8/2011 4:50 AM, Tim Whittington wrote: On Fri, Apr 8, 2011 at 2:40 AM, Christopher Schultz ch...@christopherschultz.net wrote: Mark, I understand that a fix has already been applied, but... On 4/6/2011 7:16 AM, Mark Thomas wrote: I thought of two options for issue 3: a) Assign a processor (+ inputbuffer, output buffer etc.) to a socket and don't recycle it until the socket is closed. - Increases memory requirements. - Fixes issue 3 - Retains current request processing order. b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue What about a third option that is essentially (a) except that you trim the input buffer and discard the already-processed request(s) from it. The input buffer stays bound to the request and continues where it left-off when another request processor becomes available. That would maintain scheduling fairness and hopefully not require much in the way of memory usage. Since pipelining requests with entities is not a good idea, the chances of getting a large amount of data in the input buffer is relatively low. There's also a limit on the size of that buffer, too. Would that still represent too large of a memory requirement? The input buffer is 8k by default (max header size), so this could be significant with a large maxConnections. Pruning the buffers to retain only required space would generate a lot of garbage, so probably not a good option either. Are those buffers ever discarded? I guess it comes down to whether the 8k buffer belongs to the connection or to the request. It looks like the bug arises from the buffer being treated like it belongs to the request when it really belongs to the connection. I agree, switching to a request-owned buffer strategy would certainly increase the memory footprint since you'd need a buffer for each pending request (which may be quite high when using NIO an/or async). Thanks for clarifying that. If the fairness becomes a practical problem, reducing maxKeepAliveRequests (100 by default) would force pipelining clients to the back of the queue regularly. How would this work, though? If the bug under discussion arises from a connection essentially disconnecting one of these buffers from the request whence it came, doesn't re-queuing the request re-introduce the bug? -chris signature.asc Description: OpenPGP digital signature
Re: bindOnInit and maxConnections for AJP connectors
On 4/6/2011 3:51 PM, Tim Whittington wrote: b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue +1 It's an unfair scheduling, but given issue 2 that's a fairly moot point with the BIO connector. sounds reasonable. this is how we used to do it (we may still) in APR. it avoids one context switch not needed, and in NIO we could do this too, since we can even check the network buffer without blocking best Filip - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
Are those buffers ever discarded? I guess it comes down to whether the 8k buffer belongs to the connection or to the request. It looks like the bug arises from the buffer being treated like it belongs to the request when it really belongs to the connection. I agree, switching to a request-owned buffer strategy would certainly increase the memory footprint since you'd need a buffer for each pending request (which may be quite high when using NIO an/or async). Thanks for clarifying that. If the fairness becomes a practical problem, reducing maxKeepAliveRequests (100 by default) would force pipelining clients to the back of the queue regularly. How would this work, though? If the bug under discussion arises from a connection essentially disconnecting one of these buffers from the request whence it came, doesn't re-queuing the request re-introduce the bug? I was thinking of closing the connection (as maxKeepAliveRequests) does now. The re-connect would go to the back of the queue. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
Mark, I understand that a fix has already been applied, but... On 4/6/2011 7:16 AM, Mark Thomas wrote: I thought of two options for issue 3: a) Assign a processor (+ inputbuffer, output buffer etc.) to a socket and don't recycle it until the socket is closed. - Increases memory requirements. - Fixes issue 3 - Retains current request processing order. b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue What about a third option that is essentially (a) except that you trim the input buffer and discard the already-processed request(s) from it. The input buffer stays bound to the request and continues where it left-off when another request processor becomes available. That would maintain scheduling fairness and hopefully not require much in the way of memory usage. Since pipelining requests with entities is not a good idea, the chances of getting a large amount of data in the input buffer is relatively low. There's also a limit on the size of that buffer, too. Would that still represent too large of a memory requirement? -chris signature.asc Description: OpenPGP digital signature
Re: bindOnInit and maxConnections for AJP connectors
On 05/04/2011 10:50, Tim Whittington wrote: Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) This is indeed the case. There are a number of issues with the current BIO implementation. 1. Keep-alive timeouts As per the TODO comment in Http11Processor#process(), the keep-alive timeout needs to take account of the time spent in the queue. 2. The switch to a queue does result in the possibility of requests with data being delayed by requests without data in keep-alive. 3. HTTP pipe-lining is broken (this is bug 50957 [1]). The sequence is: - client sends 1 complete request and part of second request - tomcat processes first request - tomcat recycles the input buffer - client sends remainder of second request - tomcat sees an incomplete request and returns a 505 There are variations of this depending on exactly how much of the second request has been read by Tomcat at the point the input buffer is recycled. Note that r1086349 [2] has protected against the worst of what could go wrong (mixed responses etc) but has not fixed the underlying issue. The change that triggered all of the above issues is r822234 [3]. Reverting r822234 isn't an option as the async code depends on elements of if. The fix for issue 1 is simple so I do not intend to discuss it further. The fix for issue 2 is tricky. The fundamental issue is that to resolve it and to keep maxConnections maxThreads we need NIO like behaviour from a BIO socket which just isn't possible. Fixing 1 will reduce the maximum length of delay that any one request might experience which will help but that won't address the fundamental issue. For sockets in keepalive, I considered trying to fake NIO behaviour by using a read with a timeout of 1ms, catching the SocketTimeoutException and returning them to the back of the queue if there is no data. The overhead of that looks to be around 2-3ms for a 1ms timeout. I'm worried about CPU usage but for a single thread this doesn't seem to be noticeable. More testing with multiple threads is required. The timeout could be tuned by looking at the current number of active threads, size of the queue etc. but it is an ugly hack. Returning to the pre [3] approach of disabling keep-alive once connections 75% of threads would fix this at the price of no longer being able to support maxConnections maxThreads. I thought of two options for issue 3: a) Assign a processor (+ inputbuffer, output buffer etc.) to a socket and don't recycle it until the socket is closed. - Increases memory requirements. - Fixes issue 3 - Retains current request processing order. b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue The fixes for 1 3 seem fairly straight forward and unless anyone objects, I'll go ahead (when I get a little time) and implement those. I think the fix for 2 needs some further discussion. What do folks think? Mark [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=50957 [2] http://svn.apache.org/viewvc?rev=1086349view=rev [3] http://svn.apache.org/viewvc?view=revrev=823234 cheers tim On Tue, Apr 5, 2011 at 8:51 PM, Tim Whittington t...@apache.org wrote: In the AJP standard implementation docs, the following are not mentioned, although they're properties of AbstractEndpoint and probably should work: - bindOnInit - maxConnections Am I right in assuming these should be possible in the AJP connector (my reading of the code indicates they are - just wanted to check if something arcane was going on)? If so I'll update the docs. cheers tim - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On Wed, Apr 6, 2011 at 11:16 PM, Mark Thomas ma...@apache.org wrote: On 05/04/2011 10:50, Tim Whittington wrote: Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) This is indeed the case. There are a number of issues with the current BIO implementation. 1. Keep-alive timeouts As per the TODO comment in Http11Processor#process(), the keep-alive timeout needs to take account of the time spent in the queue. 2. The switch to a queue does result in the possibility of requests with data being delayed by requests without data in keep-alive. 3. HTTP pipe-lining is broken (this is bug 50957 [1]). The sequence is: - client sends 1 complete request and part of second request - tomcat processes first request - tomcat recycles the input buffer - client sends remainder of second request - tomcat sees an incomplete request and returns a 505 There are variations of this depending on exactly how much of the second request has been read by Tomcat at the point the input buffer is recycled. Note that r1086349 [2] has protected against the worst of what could go wrong (mixed responses etc) but has not fixed the underlying issue. The change that triggered all of the above issues is r822234 [3]. Reverting r822234 isn't an option as the async code depends on elements of if. The fix for issue 1 is simple so I do not intend to discuss it further. The fix for issue 2 is tricky. The fundamental issue is that to resolve it and to keep maxConnections maxThreads we need NIO like behaviour from a BIO socket which just isn't possible. Fixing 1 will reduce the maximum length of delay that any one request might experience which will help but that won't address the fundamental issue. For sockets in keepalive, I considered trying to fake NIO behaviour by using a read with a timeout of 1ms, catching the SocketTimeoutException and returning them to the back of the queue if there is no data. The overhead of that looks to be around 2-3ms for a 1ms timeout. I'm worried about CPU usage but for a single thread this doesn't seem to be noticeable. More testing with multiple threads is required. The timeout could be tuned by looking at the current number of active threads, size of the queue etc. but it is an ugly hack. Returning to the pre [3] approach of disabling keep-alive once connections 75% of threads would fix this at the price of no longer being able to support maxConnections maxThreads. Yeah, I went down this track as well before getting to the Just use APR/NIO state of mind. It is an ugly hack, but might be workable if the timeout is large enough to stop it being a busy loop on the CPU. With 200 threads, even a 100ms timeout would give you a 'reasonable' throughput. Even if we do this, I still think maxConnections should be somewhat closer to maxThreads than it is now if the BIO connector is being used. I thought of two options for issue 3: a) Assign a processor (+ inputbuffer, output buffer etc.) to a socket and don't recycle it until the socket is closed. - Increases memory requirements. - Fixes issue 3 - Retains current request processing order. b) Check the input buffer at the end of the loop in Http11Processor#process() and process the next request if there is any data in the input buffer. - No Increase in memory requirements. - Fixes issue 3 - Pipelined requests will get processed earlier (before they would have been placed at the back of the request processing queue) I think option b) is the way to go to fix issue +1 It's an unfair scheduling, but given issue 2 that's a fairly moot point with the BIO connector. The fixes for 1 3 seem fairly straight forward and unless anyone objects, I'll go ahead (when I get a little time) and implement those. I think the fix for 2 needs some further discussion. What do folks think? Mark [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=50957 [2] http://svn.apache.org/viewvc?rev=1086349view=rev [3] http://svn.apache.org/viewvc?view=revrev=823234 cheers tim On Tue, Apr 5, 2011 at 8:51 PM, Tim Whittington t...@apache.org wrote: In the AJP standard implementation docs, the following are not mentioned, although they're properties of AbstractEndpoint and probably should work: - bindOnInit - maxConnections Am I right in assuming these should be possible in the AJP connector (my reading of the code indicates they are - just wanted to
bindOnInit and maxConnections for AJP connectors
In the AJP standard implementation docs, the following are not mentioned, although they're properties of AbstractEndpoint and probably should work: - bindOnInit - maxConnections Am I right in assuming these should be possible in the AJP connector (my reading of the code indicates they are - just wanted to check if something arcane was going on)? If so I'll update the docs. cheers tim - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
Further to this, the docs for maxConnections currently state: This setting is currently only applicable to the blocking Java connectors (AJP/HTTP). But both the NIO and APR connectors use this setting in their Acceptor implementations, limiting the number of concurrent connections before they're accepted and handed off to the pollers. It's possibly the case for the APR connector that pollerSize should be used in preference (keeping maxConnections pollerSize), but the NIO connector doesn't have a poller size config option (the connector summary describes it as 'restricted by mem'. The Connector Comparison table (and the discussion of connection/thread behaviour) in the HTTP connector docs are thus slightly misleading. Is what's actually going on more like: APR: use maxConnections == pollerSize (smallest will limit, but if pollerSize maxConnections then the socket backlog effectively won't be used as the poller will keep killing connections as they come in) NIO: use maxConnections to limit 'poller size' HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) cheers tim On Tue, Apr 5, 2011 at 8:51 PM, Tim Whittington t...@apache.org wrote: In the AJP standard implementation docs, the following are not mentioned, although they're properties of AbstractEndpoint and probably should work: - bindOnInit - maxConnections Am I right in assuming these should be possible in the AJP connector (my reading of the code indicates they are - just wanted to check if something arcane was going on)? If so I'll update the docs. cheers tim - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 5 avr. 2011, at 11:50, Tim Whittington wrote: HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) I think it was tc6 behavior, but it changed with tc7 : threads are now returned to the pool between requests, even with keep-alive connections. Sylvain - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: bindOnInit and maxConnections for AJP connectors
On 05/04/2011 19:42, Sylvain Laurent wrote: On 5 avr. 2011, at 11:50, Tim Whittington wrote: HTTP: use maxConnections. For keep alive situations, reduce maxConnections to something closer to maxThreads (the default config is 10,000 keepalive connections serviced by 200 threads with a 60 second keepalive timeout, which could lead to some large backlogs of connected sockets that take 50 minutes to get serviced) I think it was tc6 behavior Nope. tc6 behaviour was one thread per connection, including during keep-alive. , but it changed with tc7 : threads are now returned to the pool between requests, even with keep-alive connections. tc7 behaviour does return the thread to the pool but it is certainly possible for something along the lines Tim is describing to happen although I don't think the delay could be as long as 50 minutes. I'd need to double check the code for keep-alive time-outs to be sure. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
