Re: [Mod-fcgid-users] fcgid large file uploading and input buffering
Hi, Very good improvement ! But isn't it possible to disable entirely the fcgid input buffering ? Some applications, particularly PHP are already designed to handle the input processing, and if we pre-buffer the request into mod_fcgid (thanks, this isn't a DoS risk anymore :) ), features like the file uploading progress (RFC1867) can't be used : PHP will receive all the input data at the same time. While I think a small buffer like 64k for input can be useful, if the request is larger it may be better to transmit the input data directly to the FastCGI process. Looking at the code before your improvement (version 1.34), fcgid_bridge.c:591 sends a bucket to handle_request(), then handle_request() write all the input data at the same time with proc_write_ipc() on line 372. The good and simple way would be to collect input data directly into handle_request, and looping over proc_write_ipc to progressively send the data. Any problem in doing this ? This would completely avoid the need of having to handle large buffer with files. The ultimate way would be to choose with a config switch if we want input to be fully buffered or not (a small buffer like 8-64k before proc_write_ipc is still a good thing !). By the way, considering you are using apr_* functions, all of this must be thread safe, but is it really ? (for an obvious use with the worker MPM). Again : very very good work ! Gabriel Ryan Pan wrote: Hi, Please check the latest cvs tree for new source code. The new release has fix this problem. Now mod_fcgid will swap the http request to disk if it's longer than 64k :)And I added two configurations: MaxRequestLen( default 1G byte, return internal server error if http request longer than it) MaxRequestInMem( default 64k, store the request to tmp file if request longer than it) Thanks - Original Message - From: Gabriel Barazer [EMAIL PROTECTED] To: mod-fcgid-users@lists.sourceforge.net Cc: [EMAIL PROTECTED] Sent: Monday, April 30, 2007 9:21 PM Subject: fcgid large file uploading and input buffering Hello, I experienced recently some problmes since a customer is doing large file uploads with PHP (which is run by mod_fcgid, of course) : It seems mod_fcgid is consuming much memory when uploading a file to PHP. I found in the source file fcgid_bridge.c:473 the problem : as said in the source, the entire request (stdin/post input) is loaded into memory before sending it to the fastcgi Application Server (PHP in our case). Although it's a well workaround for dealing with slow clients, I think this is not the good behavior to implement, here are the points highlighted : - Uploading files is becoming a major security problem, since a DoS can be triggered by uploading a very large file (I experienced some attacks with 1/2GB over a fast connection) - Additionnally, Video (=large) file uploading is becoming more and more popular, increasing the memory consumption. - Dealing with slow clients must be done by the appliction server, which can take any appropriate measure (e.g. having a special queue processing for slow clients) - Upload progress meter is not possible if all the input data is buffered before sent to the fastcgi process. (see RFC1867 : File Upload Progress hook handler) - Upload of large files is better handled by the fast cgi AS, because of various method used to store the upload data during progress (at the application level , not the communication level that fastcgi is). e.g. PHP handles file upload by creating temporary files, which location of these can be customised by a php.ini directive. I think this task has not to be handled by the fastcgi layer (which serves as a comm./bridge protocol, not a input processor) - There is no need for the fastcgi process manager to handle and buffer slow clients : A FastCGI application designed to handle load can handle multiple connections AND the mod_fcgid process manager already does multiple connection management with the adaptive spawning feature for application which are not multi-tasked/threaded. (I even know fastcgi applications which embed a process manager themselves) What are the problems with slow clients : - Sending input is very long, not constant : e.g. with shaped connections : data flow is sent by peaks foloowed by no data input for a variable time. - Connection is longer busy at the Apache level, but at the fastcgi application level too (the workaround of buffering all the input prevent the fastcgi app from being busy buring the input loading). How to deal with this, my proposal : - What about buffering input like the output buffer, by chunks of, say, 8Kbytes ? The major problem is the time to fill the buffer : if the time required to fill the buffer is too long, application can timeout, but I think this is the normal behavior of an application to manage communication timeout. What
Re: [Mod-fcgid-users] Debian Etch: Strange delay upon first start of fcgi-application (PHP)
On 03/10/2008 3:19:16 PM +0100, Tobias Wiersch [EMAIL PROTECTED] wrote: Gabriel Barazer schrieb: 1) Is the delay *exactly* 2 seconds when calling a PHP script (before getting the http error 500 from your suexec test) ? Or is there another additionnal delay to look for ? I think it is a 1 second delay. I made a time wget http://localhost/testxxx.php; to circumvent network latency (after a apache reload). Result1: real0m1.054s Result2: real0m1.056s (unpatched suexec, php was normally loaded, script contained only an echo) Subsequent requests: Result1: real0m0.008s Result2: real0m0.009s And the results from a server running under Debian Sarge instead of Etch (the Etch server is actually much faster and the Sarge server slower - but the results for the first call are nonetheless 10 times better under Sarge!?) first call: real0m0.119s second call: real0m0.023s I'm confused. Is it the same server, same kernel other hardware/software specs ? If you try 2 different debian distrubtion, there may be some changes in PHP/libraries. What you need to do to isolate the problem is leave PHP alone because you have the start delay even without it. calling exit() within suexec or returning just before the execv is a good test case. Can you test on your 2 servers with the patched suexec? Test multiple times with apache stop or reboot commands between to be sure the delay is the same. If it's the same delay, it's likely to be a hardcoded time value; if the delay changes, it can be a library load time or a cache prefetch time. Are you installing mod_fcgid with the automated system (apt-get) or compiling from source? I know many debian users love to use the apt-get system, but this is really not what you want to do to test a specific problem in a specific module version. You have to only modify 1 thing between your tests or you won't be able to isolate the problem. It's like you have a cold AND headaches, and you take 2 different pills: you won't know which one made your headaches disappear. Oh, and add something like -s 4096 to the strace command to see more on the socket read. Strace with -s 4096: nothing relevant here Gabriel - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Mod-fcgid-users mailing list Mod-fcgid-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mod-fcgid-users
Re: [Mod-fcgid-users] Debian Etch: Strange delay upon first start of fcgi-application (PHP)
On 03/10/2008 7:53:10 PM +0100, Tobias Wiersch [EMAIL PROTECTED] wrote: Yes, the results were from 2 servers. The Etch-server has the problem. I wrote the results from the sarge server only to show the difference. can you tell what is the mod_fcgid version on each server? If the version is the same, you may be facing the same bug on each server, but for some reason it's not triggered on Debian Sarge. Anyway, if it's a bug and we have to relase an update, you'll have to compile from source and not from packages... What you need to do to isolate the problem is leave PHP alone because you have the start delay even without it. calling exit() within suexec or returning just before the execv is a good test case. Can you test on your 2 servers with the patched suexec? Test multiple times with apache stop or reboot commands between to be sure the delay is the same. Ok, on Etch: real0m1.010s (always the same because the fastcgi-link to php can't be established because of the exit() at the beginning of suexec) very good to know, except if the module on etch is an old version. Have you set the same fcgid directives in your apache configuration? On Sarge: real0m0.005s (same patched suexec) Are you installing mod_fcgid with the automated system (apt-get) or compiling from source? I use the standard-debian package because I don't want to re-compile the packages after every security fix. Do you think the problem is fixed in the later releases? AFAIK Etch is newer than Sarge (I'm *really* not a Debian expert, made my own OS), so the fcgid module is probably newer as well. Last version is 2.2 and some hardcoded variables related to the scoring (the mechanism which determines if a new fastcgi application has to be spawned) have been changed. BTW, have you got anything related to mod_fcgid in your error log? You have to log some info or debug level errors in your apache config. Maybe you will find some interesting stuff. Gabriel - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Mod-fcgid-users mailing list Mod-fcgid-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mod-fcgid-users
Re: [Mod-fcgid-users] Limit maximum number of processes
Hi On 07/15/2008 5:45:54 PM +0200, Ron [EMAIL PROTECTED] wrote: IdleTimeout 10 IdleScanInterval 5 MaxProcessCount 0 DefaultMaxClassProcessCount 1 SpawnScoreUpLimit 0 Setting values to 0 cannot make mod_fcgid behave as expected. More on this below. The browser opens this script 2-2-2 etc (possibly due to the maxSimultanousPersistentConnections setting in browsers). When we open this script however in multiple browsers a fatal DOS-like number of processes is forked, even when we use the settings above (MaxProcessCount = 0). You have to use MaxProcessCount = 1 to limit to one process. Using 0 is not really handled, so I think it could mean unlimited. We understand that each 'main fcgi process' has 4 children, so 5 processes in total. There appears however to be one 'main fcgi process' per (running) browser without a limit. When we close all browsers and neatly run the iframe-page in only one browser, then no more extra processes are forked. Setting PHP children is completely independant of the fcgid module. So mod_fcgid will only rely on the processes it spawned itself, not the ones PHP spawned afterwards. How can we prevent that this many processes are forked? Ideally we want only a maximum of one 'main fcgi process'. Try setting MaxClassProcessCount to 1 in the global scope (not inside a VirtualHost). I'm not sure how local scopes are handled/merged with the global config. What you are really looking for is not a dynamic process spawning, but a static process handling like the old mod_fastcgi behavior. This method is not available in mod_fcgid, so you may have to use mod_fastcgi instead and switch to mod_fcgid later. Some background information is that we also use the module xcache. This module loads php-opcode in a mmap shared between forked processes. So each time a new 'main fcgi process' is created the xcache re-caches all opcode for the php pages. When only one 'main fcgi process' with its children stays running the xcache also remains in memory for optimal performance. This doesn't really matters. You will have a duplicated cache and a bit of memory waste when spawning multiple main processes, but except if you are very tight on memory, that is not really a problem. In fact, by doing this, you are reducing the pressure on the xcache shared memory area because you are spreading the read and write locks onto multiple independant areas. (we also use suexec if that matters) You should always use it :) Using PHP without suexec shouldn't even be possible, given how big the security hole is without suexec and with multiple PHP users. Gabriel - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ Mod-fcgid-users mailing list Mod-fcgid-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mod-fcgid-users
