On Wed, Feb 18, 2009 at 09:39:31PM +0100, Ruediger Pluem wrote:
> On 02/18/2009 11:16 AM, Joe Orton wrote:
> There is still a nasty issue with the trunk code that can cause you to
> run out of FD's as the new non blocking core output filter has some trouble
> setting aside the file buckets to the correct pool (it puts them in the
> connection pool which is too long living and bad).
>
> See thread starting at
>
> http://mail-archives.apache.org/mod_mbox/httpd-dev/200901.mbox/%[email protected]%3e
>
> I admit that I lost momentum on this, but this definitely needs to be fixed.
Yes, that looks broken.
Something like the patch below should fix it, though I'm not sure it's
really a good idea to be allowing a pathological case of 16 fds consumed
per client. It might present some interesting tuning challenges;
hitting fd limits before hitting num(client) limits.
Index: core_filters.c
===================================================================
--- core_filters.c (revision 734690)
+++ core_filters.c (working copy)
@@ -367,6 +367,7 @@
#define THRESHOLD_MIN_WRITE 4096
#define THRESHOLD_MAX_BUFFER 65536
+#define THRESHOLD_MAX_FILES 16
/* Optional function coming from mod_logio, used for logging of output
* traffic
@@ -380,7 +381,7 @@
core_output_filter_ctx_t *ctx = net->out_ctx;
apr_bucket_brigade *bb;
apr_bucket *bucket, *next;
- apr_size_t bytes_in_brigade, non_file_bytes_in_brigade;
+ apr_size_t bytes_in_brigade, non_file_bytes_in_brigade,
num_files_in_brigade;
/* Fail quickly if the connection has already been aborted. */
if (c->aborted) {
@@ -466,6 +467,8 @@
bytes_in_brigade = 0;
non_file_bytes_in_brigade = 0;
+ num_files_in_brigade = 0;
+
for (bucket = APR_BRIGADE_FIRST(bb); bucket != APR_BRIGADE_SENTINEL(bb);
bucket = next) {
next = APR_BUCKET_NEXT(bucket);
@@ -497,13 +500,17 @@
next = APR_BUCKET_NEXT(bucket);
}
bytes_in_brigade += bucket->length;
- if (!APR_BUCKET_IS_FILE(bucket)) {
+ if (APR_BUCKET_IS_FILE(bucket)) {
+ num_files_in_brigade++;
+ }
+ else {
non_file_bytes_in_brigade += bucket->length;
}
}
}
- if (non_file_bytes_in_brigade >= THRESHOLD_MAX_BUFFER) {
+ if (non_file_bytes_in_brigade >= THRESHOLD_MAX_BUFFER
+ || num_files_in_brigade >= THRESHOLD_MAX_FILES) {
/* ### Writing the entire brigade may be excessive; we really just
* ### need to send enough data to be under THRESHOLD_MAX_BUFFER.
*/
