Re: [Mongrel] Rails' send_file, Mongrel, and *gasp* memory
Hi Jon,
I don't know much about mod_secdownload, but I do know that both
apache2.x.x and lighttpd can use the X-Sendfile header.
See: http://celebnamer.celebworld.ws/stuff/mod_xsendfile/
I have an Apache - Mongrel setup where I've installed mod_xsendfile for
apache. When you want to send a file to the user
simply do something like:
response.headers['Content-Type'] = application/force-download
response.headers['Content-Disposition'] = attachment;
filename=\#{file.short_name}\
response.headers[X-Sendfile] = file.name
response.headers['Content-length'] = file.size_in_bytes
apache will then step in and serve the file.
Regards,
Steven
Jonathan Leighton wrote:
On Fri, 2006-12-22 at 00:06 -0800, Zed A. Shaw wrote:
To be clear, send_file does not read the whole file into memory -- it blows
chunks into the output stream. Mongrel and pure-Ruby FastCGI use StringIO to
buffer the response before sending it to the client.
X-Sendfile is definitely the way to go in any case.
Just to be extra clear, neither of the two main systems for hosting Rails
would need to buffer the output if Rails didn't alternate between sending
headers and body content.
And yes, X-Sendfile is the way to go.
Thanks for all your responses. Just in case it makes a difference, I'll
clarify about my setup. Basically we have an Apache on port 80 which
can't be removed from the equation. The Rails app is served directly
Apache - Mongrel, and now I have made it so the /file/ path is proxied
Apache - lighttpd, which serves the files using mod_secdownload. I
wasn't actually aware of X-Sendfile so that's interesting -- is this
better, worse or equal to using mod_secdownload? I wouldn't have thought
it made a difference as they both enable lighttpd to serve the static
files, but if people think I should drop the mod_secdownload solution in
favour of X-Sendfile I guess I could set up the application to go
through lighttpd too.
Thanks again,
Jon
___
Mongrel-users mailing list
Mongrel-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-users
___
Mongrel-users mailing list
Mongrel-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Rails' send_file, Mongrel, and *gasp* memory
On Dec 21, 2006, at 2:15 PM, Jonathan Leighton wrote: I've had a right fun few days at work trying to figure out why our Rails app (which isn't under very heavy load) kept eating memory and bringing our server to our knees. Eventually I traced it to send_file (which was in a way a relief as it wasn't down to my coding ;) -- every time a user started downloading, the memory consumed by the app would jump, and wouldn't go down again, even when they had cancelled the download (I haven't tested whether the memory would still be consumed after they completed the download). This has become apparent recently as the app has been used for larger files (150MB and upwards). As a fix I've delegated file serving to lighttpd + mod_sec_download, not quite as convenient but probably better in the long run anyway. So the problem is solved, but I was just wondering why this happened in the first place; according to the Rails docs, send_file buffers the response in order to not tie up memory. However, this was clearly not happening -- is this possibly related to how Mongrel interacts with Rails? If so, I gave it a try under lighttpd and still had the problem, any idea why? I'm really just looking to understand the issue better as I hate to walk away from a problem without fully comprehending it :) Thanks, -- Jonathan Leighton, Web Developer Hey - The rails send_file method is not the same thing as setting an X- SendFile header. If you are already using lighttpd for your server then your in good shape. THe rails send_file method actually does read the whole file into memory as it sends it to the client. SO you will see memory problems like you have if you use it on large files. The nice way to do it is to use the X-SendFile header in lighty. In a rails action that just wants to serve a static file from somewhere off the filesystem you can make it look like this: def download file = File.find params[:file] # or whatever response.headers[X-Sendfile] = file.path render :nothing = true end When you enable X-SendFIle support in lighty it will see this empty response with just headers set and lighty will serve the static file fast. Here is a good blog post about something similar: http://blog.craz8.com/articles/2006/11/13/lighttpd-proxy-with-x- sendfile-and-action_cache CHeers- -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- [EMAIL PROTECTED] -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273) ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Rails' send_file, Mongrel, and *gasp* memory
On 12/21/06, Ezra Zygmuntowicz [EMAIL PROTECTED] wrote: The rails send_file method is not the same thing as setting an X- SendFile header. If you are already using lighttpd for your server then your in good shape. THe rails send_file method actually does read the whole file into memory as it sends it to the client. SO you will see memory problems like you have if you use it on large files. The nice way to do it is to use the X-SendFile header in lighty. In a rails action that just wants to serve a static file from somewhere off the filesystem you can make it look like this: To be clear, send_file does not read the whole file into memory -- it blows chunks into the output stream. Mongrel and pure-Ruby FastCGI use StringIO to buffer the response before sending it to the client. X-Sendfile is definitely the way to go in any case. jeremy ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] Rails' send_file, Mongrel, and *gasp* memory
I've had a right fun few days at work trying to figure out why our Rails app (which isn't under very heavy load) kept eating memory and bringing our server to our knees. Eventually I traced it to send_file (which was in a way a relief as it wasn't down to my coding ;) -- every time a user started downloading, the memory consumed by the app would jump, and wouldn't go down again, even when they had cancelled the download (I haven't tested whether the memory would still be consumed after they completed the download). This has become apparent recently as the app has been used for larger files (150MB and upwards). As a fix I've delegated file serving to lighttpd + mod_sec_download, not quite as convenient but probably better in the long run anyway. So the problem is solved, but I was just wondering why this happened in the first place; according to the Rails docs, send_file buffers the response in order to not tie up memory. However, this was clearly not happening -- is this possibly related to how Mongrel interacts with Rails? If so, I gave it a try under lighttpd and still had the problem, any idea why? I'm really just looking to understand the issue better as I hate to walk away from a problem without fully comprehending it :) You don't say it, but are you using the sendfile gem? There was a lot of talk about there being serious problems with the sendfile gem on various platforms to the point that Zed explicitly told people to stop using it Just a thought. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
