Since a vanilla Mongrel will store uploaded data on disk in a tempfile, maybe it has something to do with your Railsdav plugin?
I just tested an 80 MB upload with my Apache 2.2.6 reverse proxy and Mongrel 1.0.1 cluster and it does save the object first to /tmp on disk and not in memory. The memory used by the mongrel process before I started uploading was 50 MB. While uploading it peaked at 57 MB but was mostly stable around 50-52 MB. And after the upload was done it was back again at 50 MB. Maybe your $TMPDIR is not writable for the user you are running mongrel under? Just before the upload was finished: $ ls -hl /tmp/ total 160288 -rw------- 1 mongrel wheel 78.2M Oct 15 16:31 mongrel.25.0 I would start looking in the Railsdav plugin you are using, seems like they changed the default of saving to disk, to memory. Cheers. ----- Original Message ---- From: Daniel Brahneborg <[EMAIL PROTECTED]> To: [email protected] Sent: Monday, October 15, 2007 4:10:06 PM Subject: Re: [Mongrel] POST with huge HTTP body Thanks for the replies. The fact that Mongrel shouldn't be the first one to get the POST/PUT data was a good point that I hadn't thought of. However, even when talking directly to Mongrel, the memory consumption of the application increases until it crashes if too much data is sent. In my case, I was sending a 500MB file from a Webdav client. At some point all of it was loaded into RAM, which doesn't work. Are there any more productive alternatives than writing the entire thing from scratch in C? Please? :) /Daniel On 10/15/07, Tim Kuijsten <[EMAIL PROTECTED]> wrote: > A while ago I wrote a plugin to limit uploads in Mongrel. Mongrel does save uploads to a tempfile on disk - if the upload is bigger than about 12 KB - using a Ruby TempFile-object. (which stores in the $TMPDIR, /tmp on most systems). > > > The request is handed over to Rails after it's fully received by Mongrel. > > > I'm not sure if this "saving to disk" works the same with chunked uploads (uploads without a Content-Length header). > > > > Maybe my plugin can help you: http://slasaus.netsend.nl/articles/show/7 (warning: it works, but is not very elegant). > > > Gl, > > > Tim > > ----- Original Message ---- > From: Daniel Brahneborg <[EMAIL PROTECTED]> > To: [email protected] > Sent: Monday, October 15, 2007 2:00:20 PM > Subject: [Mongrel] POST with huge HTTP body > > > Hi, > > I found the following page describing how to stream data from the > server to the client: > > http://api.rubyonrails.com/classes/ActionController/Streaming.html > > I want to do the opposite, streaming data from the client to the > server, > letting the controller saving the data while it's being received. > In particular, I want to upload large files to the RailsDav > (http://www.liverail.net/railsdav) plugin, without having to store > the entire file in memory during the operation. > > This seems to be a Mongrel problem, so I started looking at this > plugin: > > http://mongrel.rubyforge.org/docs/upload_progress.html > > How do I get Mongrel to store the incoming POST data in a tempfile, > passing a handler to that file as the raw_post field in the Rails > request? > Is it even possible? > How is the "request" parameter to the process() method related to the > "request" object in the Rails controllers? > > /Daniel > _______________________________________________ > Mongrel-users mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/mongrel-users > > > > > > ____________________________________________________________________________________ > Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 > > > _______________________________________________ > Mongrel-users mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/mongrel-users > _______________________________________________ Mongrel-users mailing list [email protected] http://rubyforge.org/mailman/listinfo/mongrel-users ____________________________________________________________________________________ Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC _______________________________________________ Mongrel-users mailing list [email protected] http://rubyforge.org/mailman/listinfo/mongrel-users
