On 02/19/2009 12:32 PM, Joe Orton wrote: > On Wed, Feb 18, 2009 at 09:39:31PM +0100, Ruediger Pluem wrote: >> On 02/18/2009 11:16 AM, Joe Orton wrote: >> There is still a nasty issue with the trunk code that can cause you to >> run out of FD's as the new non blocking core output filter has some trouble >> setting aside the file buckets to the correct pool (it puts them in the >> connection pool which is too long living and bad). >> >> See thread starting at >> >> http://mail-archives.apache.org/mod_mbox/httpd-dev/200901.mbox/%3c495ff1b9.9070...@kippdata.de%3e >> >> I admit that I lost momentum on this, but this definitely needs to be fixed. > > Yes, that looks broken. > > Something like the patch below should fix it, though I'm not sure it's > really a good idea to be allowing a pathological case of 16 fds consumed > per client. It might present some interesting tuning challenges; > hitting fd limits before hitting num(client) limits. > > Index: core_filters.c > =================================================================== > --- core_filters.c (revision 734690) > +++ core_filters.c (working copy) > @@ -367,6 +367,7 @@ > > #define THRESHOLD_MIN_WRITE 4096 > #define THRESHOLD_MAX_BUFFER 65536 > +#define THRESHOLD_MAX_FILES 16 > > /* Optional function coming from mod_logio, used for logging of output > * traffic > @@ -380,7 +381,7 @@ > core_output_filter_ctx_t *ctx = net->out_ctx; > apr_bucket_brigade *bb; > apr_bucket *bucket, *next; > - apr_size_t bytes_in_brigade, non_file_bytes_in_brigade; > + apr_size_t bytes_in_brigade, non_file_bytes_in_brigade, > num_files_in_brigade; > > /* Fail quickly if the connection has already been aborted. */ > if (c->aborted) { > @@ -466,6 +467,8 @@ > > bytes_in_brigade = 0; > non_file_bytes_in_brigade = 0; > + num_files_in_brigade = 0; > + > for (bucket = APR_BRIGADE_FIRST(bb); bucket != APR_BRIGADE_SENTINEL(bb); > bucket = next) { > next = APR_BUCKET_NEXT(bucket); > @@ -497,13 +500,17 @@ > next = APR_BUCKET_NEXT(bucket); > } > bytes_in_brigade += bucket->length; > - if (!APR_BUCKET_IS_FILE(bucket)) { > + if (APR_BUCKET_IS_FILE(bucket)) { > + num_files_in_brigade++; > + } > + else { > non_file_bytes_in_brigade += bucket->length; > } > } > } > > - if (non_file_bytes_in_brigade >= THRESHOLD_MAX_BUFFER) { > + if (non_file_bytes_in_brigade >= THRESHOLD_MAX_BUFFER > + || num_files_in_brigade >= THRESHOLD_MAX_FILES) {
If the 16 FD's were split over more then one brigade and the brigades before us were set aside the FD's belong already to the wrong pool (the connection pool). Deleting a file bucket doesn't close the FD it uses. Regards Rüdiger
