Hi,
mod_fcgid buffer http request, because:
1. Decrease the number of FastCGI process, because all FastCGI processes don't
have to wait for the total http request.
2. mod_fcgid is able to retry another FastCGI process, if read no respond from
one FastCGI process(and kick out this process, read no respond happens a lots
on heavy load sites, many complains about mod_fastcgi's "first read error"...).
Yes, " features like the file uploading progress (RFC1867) can't be used :
PHP will receive all the input data at the same time.", it's really a problem,
let me think about it...Thank you for your great idea
Thanks
----- Original Message -----
From: "Gabriel Barazer" <[EMAIL PROTECTED]>
To: "Ryan Pan" <[EMAIL PROTECTED]>
Cc: <mod-fcgid-users@lists.sourceforge.net>; <[EMAIL PROTECTED]>
Sent: Wednesday, July 11, 2007 7:25 PM
Subject: Re: fcgid large file uploading and input buffering
> Hi,
>
> Very good improvement ! But isn't it possible to disable entirely the
> fcgid input buffering ? Some applications, particularly PHP are already
> designed to handle the input processing, and if we pre-buffer the
> request into mod_fcgid (thanks, this isn't a DoS risk anymore :) ),
> features like the file uploading progress (RFC1867) can't be used : PHP
> will receive all the input data at the same time.
>
> While I think a small buffer like 64k for input can be useful, if the
> request is larger it may be better to transmit the input data directly
> to the FastCGI process.
>
> Looking at the code before your improvement (version 1.34),
> fcgid_bridge.c:591 sends a bucket to handle_request(), then
> handle_request() write all the input data at the same time with
> proc_write_ipc() on line 372. The good and simple way would be to
> collect input data directly into handle_request, and looping over
> proc_write_ipc to progressively send the data. Any problem in doing this
> ? This would completely avoid the need of having to handle large buffer
> with files.
>
> The ultimate way would be to choose with a config switch if we want
> input to be fully buffered or not (a small buffer like 8-64k before
> proc_write_ipc is still a good thing !).
>
> By the way, considering you are using apr_* functions, all of this must
> be thread safe, but is it really ? (for an obvious use with the worker MPM).
>
> Again : very very good work !
>
> Gabriel
>
> Ryan Pan wrote:
>> Hi,
>> Please check the latest cvs tree for new source code. The new release
>> has fix this problem.
>> Now mod_fcgid will swap the http request to disk if it's longer than 64k
>> :) And I added two configurations:
>> MaxRequestLen( default 1G byte, return internal server error if http
>> request longer than it)
>> MaxRequestInMem( default 64k, store the request to tmp file if request
>> longer than it)
>>
>> Thanks
>>
>> ----- Original Message -----
>> From: "Gabriel Barazer" <[EMAIL PROTECTED]>
>> To: <mod-fcgid-users@lists.sourceforge.net>
>> Cc: <[EMAIL PROTECTED]>
>> Sent: Monday, April 30, 2007 9:21 PM
>> Subject: fcgid large file uploading and input buffering
>>
>>
>>> Hello,
>>>
>>> I experienced recently some problmes since a customer is doing large
>>> file uploads with PHP (which is run by mod_fcgid, of course) : It seems
>>> mod_fcgid is consuming much memory when uploading a file to PHP. I found
>>> in the source file fcgid_bridge.c:473 the problem : as said in the
>>> source, the entire request (stdin/post input) is loaded into memory
>>> before sending it to the fastcgi Application Server (PHP in our case).
>>> Although it's a well workaround for dealing with slow clients, I think
>>> this is not the good behavior to implement, here are the points
>>> highlighted :
>>> - Uploading files is becoming a major security problem, since a DoS can
>>> be triggered by uploading a very large file (I experienced some attacks
>>> with 1/2GB over a fast connection)
>>> - Additionnally, Video (=large) file uploading is becoming more and more
>>> popular, increasing the memory consumption.
>>> - Dealing with slow clients must be done by the appliction server, which
>>> can take any appropriate measure (e.g. having a special queue processing
>>> for slow clients)
>>> - Upload progress meter is not possible if all the input data is
>>> buffered before sent to the fastcgi process. (see RFC1867 : File Upload
>>> Progress hook handler)
>>> - Upload of large files is better handled by the fast cgi AS, because of
>>> various method used to store the upload data during progress (at the
>>> application level , not the communication level that fastcgi is). e.g.
>>> PHP handles file upload by creating temporary files, which location of
>>> these can be customised by a php.ini directive. I think this task has
>>> not to be handled by the fastcgi layer (which serves as a comm./bridge
>>> protocol, not a input processor)
>>> - There is no need for the fastcgi process manager to handle and buffer
>>> slow clients : A FastCGI application designed to handle load can handle
>>> multiple connections AND the mod_fcgid process manager already does
>>> multiple connection management with the adaptive spawning feature for
>>> application which are not multi-tasked/threaded. (I even know fastcgi
>>> applications which embed a process manager themselves)
>>>
>>>
>>> What are the problems with slow clients :
>>> - Sending input is very long, not constant : e.g. with shaped
>>> connections : data flow is sent by "peaks" foloowed by no data input for
>>> a variable time.
>>> - Connection is longer busy at the Apache level, but at the fastcgi
>>> application level too (the workaround of buffering all the input prevent
>>> the fastcgi app from being busy buring the input loading).
>>>
>>> How to deal with this, my proposal :
>>> - What about buffering input like the output buffer, by chunks of, say,
>>> 8Kbytes ? The major problem is the time to fill the buffer : if the time
>>> required to fill the buffer is too long, application can timeout, but I
>>> think this is the normal behavior of an application to manage
>>> communication timeout. What about don't buffering the input at all ?
>>> This way the data flow AND the data flow rate can by processed by the
>>> application (such as measuring the data flow rate to put a slow request
>>> in a special queue).
>>> - Because maybe some users prefer the current behavior of buffering all
>>> the input data, a compatibility switch would be a nice thing (e.g.
>>> InputBuffering Off / On)
>>>
>>> What do you think about it ?
>>>
>>> BTW: who are the current maintainer(s) of this project ? The
>>> documentation of this project is not very up-to-date and I had to read
>>> the source code to know all the directives... Maybe can I be of some help ?
>>>
>>> Regards,
>>>
>>> Gabriel
>>>
>>>
>>>
>
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Mod-fcgid-users mailing list
Mod-fcgid-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-fcgid-users
