mod_fcgid buffer http request, because: 
1. Decrease the number of FastCGI process, because all FastCGI processes don't 
have to wait for the total http request.
2. mod_fcgid is able to retry another FastCGI process, if read no respond from 
one FastCGI process(and kick out this process, read no respond happens a lots 
on heavy load sites, many complains about mod_fastcgi's "first read error"...).

    Yes,  " features like the file uploading progress (RFC1867) can't be used : 
PHP will receive all the input data at the same time.",  it's really a problem, 
let me think about it...Thank you for your great idea


----- Original Message ----- 
From: "Gabriel Barazer" <[EMAIL PROTECTED]>
To: "Ryan Pan" <[EMAIL PROTECTED]>
Cc: <mod-fcgid-users@lists.sourceforge.net>; <[EMAIL PROTECTED]>
Sent: Wednesday, July 11, 2007 7:25 PM
Subject: Re: fcgid large file uploading and input buffering

> Hi,
> Very good improvement ! But isn't it possible to disable entirely the 
> fcgid input buffering ? Some applications, particularly PHP are already 
> designed to handle the input processing, and if we pre-buffer the 
> request into mod_fcgid (thanks, this isn't a DoS risk anymore :) ), 
> features like the file uploading progress (RFC1867) can't be used : PHP 
> will receive all the input data at the same time.
> While I think a small buffer like 64k for input can be useful, if the 
> request is larger it may be better to transmit the input data directly 
> to the FastCGI process.
> Looking at the code before your improvement (version 1.34), 
> fcgid_bridge.c:591 sends a bucket to handle_request(), then 
> handle_request() write all the input data at the same time with 
> proc_write_ipc() on line 372. The good and simple way would be to 
> collect input data directly into handle_request, and looping over 
> proc_write_ipc to progressively send the data. Any problem in doing this 
> ? This would completely avoid the need of having to handle large buffer 
> with files.
> The ultimate way would be to choose with a config switch if we want 
> input to be fully buffered or not (a small buffer like 8-64k before 
> proc_write_ipc is still a good thing !).
> By the way, considering you are using apr_* functions, all of this must 
> be thread safe, but is it really ? (for an obvious use with the worker MPM).
> Again : very very good work !
> Gabriel
> Ryan Pan wrote:
>> Hi,
>>     Please check the latest cvs tree for new source code. The new release 
>> has fix this problem. 
>>     Now mod_fcgid will swap the http request to disk if it's longer than 64k 
>> :)    And I added two configurations: 
>>     MaxRequestLen( default 1G byte, return internal server error if http 
>> request longer than it)
>>     MaxRequestInMem( default 64k, store the request to tmp file if request 
>> longer than it)
>> Thanks
>> ----- Original Message ----- 
>> From: "Gabriel Barazer" <[EMAIL PROTECTED]>
>> To: <mod-fcgid-users@lists.sourceforge.net>
>> Sent: Monday, April 30, 2007 9:21 PM
>> Subject: fcgid large file uploading and input buffering
>>> Hello,
>>> I experienced recently some problmes since a customer is doing large 
>>> file uploads with PHP (which is run by mod_fcgid, of course) : It seems 
>>> mod_fcgid is consuming much memory when uploading a file to PHP. I found 
>>> in the source file fcgid_bridge.c:473 the problem : as said in the 
>>> source, the entire request (stdin/post input) is loaded into memory 
>>> before sending it to the fastcgi Application Server (PHP in our case). 
>>> Although it's a well workaround for dealing with slow clients, I think 
>>> this is not the good behavior to implement, here are the points 
>>> highlighted :
>>> - Uploading files is becoming a major security problem, since a DoS can 
>>> be triggered by uploading a very large file (I experienced some attacks 
>>> with 1/2GB over a fast connection)
>>> - Additionnally, Video (=large) file uploading is becoming more and more 
>>> popular, increasing the memory consumption.
>>> - Dealing with slow clients must be done by the appliction server, which 
>>> can take any appropriate measure (e.g. having a special queue processing 
>>> for slow clients)
>>> - Upload progress meter is not possible if all the input data is 
>>> buffered before sent to the fastcgi process. (see RFC1867 : File Upload 
>>> Progress hook handler)
>>> - Upload of large files is better handled by the fast cgi AS, because of 
>>> various method used to store the upload data during progress (at the 
>>> application level , not the communication level that fastcgi is). e.g. 
>>> PHP handles file upload by creating temporary files, which location of 
>>> these can be customised by a php.ini directive. I think this task has 
>>> not to be handled by the fastcgi layer (which serves as a comm./bridge 
>>> protocol, not a input processor)
>>> - There is no need for the fastcgi process manager to handle and buffer 
>>> slow clients : A FastCGI application designed to handle load can handle 
>>> multiple connections AND the mod_fcgid process manager already does 
>>> multiple connection management with the adaptive spawning feature for 
>>> application which are not multi-tasked/threaded. (I even know fastcgi 
>>> applications which embed a process manager themselves)
>>> What are the problems with slow clients :
>>> - Sending input is very long, not constant : e.g. with shaped 
>>> connections : data flow is sent by "peaks" foloowed by no data input for 
>>> a variable time.
>>> - Connection is longer busy at the Apache level, but at the fastcgi 
>>> application level too (the workaround of buffering all the input prevent 
>>> the fastcgi app from being busy buring the input loading).
>>> How to deal with this, my proposal :
>>> - What about buffering input like the output buffer, by chunks of, say, 
>>> 8Kbytes ? The major problem is the time to fill the buffer : if the time 
>>> required to fill the buffer is too long, application can timeout, but I 
>>> think this is the normal behavior of an application to manage 
>>> communication timeout. What about don't buffering the input at all ? 
>>> This way the data flow AND the data flow rate can by processed by the 
>>> application (such as measuring the data flow rate to put a slow request 
>>> in a special queue).
>>> - Because maybe some users prefer the current behavior of buffering all 
>>> the input data, a compatibility switch would be a nice thing (e.g. 
>>> InputBuffering Off / On)
>>> What do you think about it ?
>>> BTW: who are the current maintainer(s) of this project ? The 
>>> documentation of this project is not very up-to-date and I had to read 
>>> the source code to know all the directives... Maybe can I be of some help ?
>>> Regards,
>>> Gabriel
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
Mod-fcgid-users mailing list

Reply via email to