Re: [Mongrel] failed to allocate memory while downloading large files
Daniel Cadenas wrote: Luis Lavena wrote: please read about streaming data without loading all the file contents. You will need to either use rack directly or create a mongrel handler to serve these files outside rails. Rails can only deliver one full request meaning it will load the full file before completing. Notice that now you can use send_file which is now (Rails 3) implemented through a rack middleware and includes X-sendfile headers. yaa Daniel i already imlemented send_file method and now i downloaded 7 gb of file without fail .. The only problem am facing is how do i delete mongrel temporary after upload large files.. -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Luis Lavena wrote: On Tue, Sep 14, 2010 at 2:49 AM, Amit Tomar li...@ruby-forum.com wrote: Luis ,this is how am downloading my files send_data(@data,:filename = @containerformat.name+extension, ? ? ? ?:disposition = 'attachment') but i am getting error failed to allocate memory for large files ,for small files it's working fine Because send_data will try to load the whole file in memory. http://apidock.com/rails/ActionController/Streaming/send_data Since the file is not dynamically generated, serve it directly, don't put it behind Rails. and one thing more i observed while uploading large files , whenever i upload data more than 2gb from IE OR from mozila ,both restarts but for google chorme its working fine??? Neither IE or Mozilla do chunked uploads. Chrome does and mongrels properly streams the chunked upload to a temporary file. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 thanks luis but could you please explain how i serve it directly and for my application i can'nt use IE because as you said it don't do chunked upload. thanks again -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
On Tue, Sep 14, 2010 at 9:21 AM, Amit Tomar li...@ruby-forum.com wrote: thanks luis but could you please explain how i serve it directly Place the file inside public, provide the URL to the file as a link, your browser will start download once the file is served. send_data is not recommended for big file serving. and for my application i can'nt use IE because as you said it don't do chunked upload. Can't help you on that, please research about it. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
On Tue, Sep 14, 2010 at 2:49 AM, Amit Tomar li...@ruby-forum.com wrote: Luis ,this is how am downloading my files send_data(@data,:filename = @containerformat.name+extension, :disposition = 'attachment') but i am getting error failed to allocate memory for large files ,for small files it's working fine Because send_data will try to load the whole file in memory. http://apidock.com/rails/ActionController/Streaming/send_data Since the file is not dynamically generated, serve it directly, don't put it behind Rails. and one thing more i observed while uploading large files , whenever i upload data more than 2gb from IE OR from mozila ,both restarts but for google chorme its working fine??? Neither IE or Mozilla do chunked uploads. Chrome does and mongrels properly streams the chunked upload to a temporary file. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Luis Lavena wrote: On Tue, Sep 14, 2010 at 9:21 AM, Amit Tomar li...@ruby-forum.com wrote: thanks luis but could you please explain how i serve ?it directly Place the file inside public, provide the URL to the file as a link, your browser will start download once the file is served. send_data is not recommended for big file serving. and for my application i can'nt use ?IE because as you said it don't ?do chunked upload. Can't help you on that, please research about it. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 sorry luis but i didn't get you i uploaded my file in filesystem than how can i place file in public folder could you please give some example?? -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
On Tue, Sep 14, 2010 at 9:39 AM, Amit Tomar li...@ruby-forum.com wrote: sorry luis but i didn't get you i uploaded my file in filesystem than how can i place file in public folder could you please give some example?? Are you using a plugin to manage your uploads? How are you saving the file? Take the temporary file and use File.move or FileUtils.mv, wonder what are you doing right now for that. Please use google to search, there are plenty of file uploading tutorials about Ruby on Rails on the internet covering do from scratch or using gem/plugins. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Luis Lavena wrote: On Tue, Sep 14, 2010 at 9:39 AM, Amit Tomar li...@ruby-forum.com wrote: sorry luis but i didn't get you i uploaded my file in filesystem than how ?can ?i place file in public folder could you please give some example?? Are you using a plugin to manage your uploads? How are you saving the file? Take the temporary file and use File.move or FileUtils.mv, wonder what are you doing right now for that. Please use google to search, there are plenty of file uploading tutorials about Ruby on Rails on the internet covering do from scratch or using gem/plugins. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 this is how am uploading def upload(params) path = get_path(params) File.open(path, wb) do |f| while buff= params[:location].read(4096) f.write(buff) end end end def get_path(params) name = params[:location].original_filename directory = //192.168.247.77/Smruti/streams path = File.join(directory, name) return path end def copyFile(uploaded_file) @uploaded_file = uploaded_file directory = //192.168.147.17/Smruti/streams #url_dir = data url_dir = //192.168.147.17/Smruti/streams name = File.basename(@uploaded_file) path = File.join(directory, name) if (!FileTest.exist?(path)) FileUtils.copy @uploaded_file, path else return duplicate end url = File.join(url_dir, name) return url end end and after copying it filesystem -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Amit Tomar wrote: Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 luis i put file in public folder but now i have four copies of same file one is mongel temp file ,2nd cgi file,3rd in filesyste and 4th in public for large don't you think its not efficient approch ?? -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
On Tue, Sep 14, 2010 at 9:50 AM, Amit Tomar li...@ruby-forum.com wrote: this is how am uploading def upload(params) path = get_path(params) File.open(path, wb) do |f| while buff= params[:location].read(4096) f.write(buff) end end end That is copying the file every 4k for you, you can simply File.move form the temporay filename to the final name in your remote location. It is expected you dispose the temporary file when you're done with it. Please see how proper file uploading and processing is done with Ruby on Rails. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Luis Lavena wrote: On Tue, Sep 14, 2010 at 9:50 AM, Amit Tomar li...@ruby-forum.com wrote: ? ? ? ?end end That is copying the file every 4k for you, you can simply File.move form the temporay filename to the final name in your remote location. It is expected you dispose the temporary file when you're done with it. Please see how proper file uploading and processing is done with Ruby on Rails. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 Thanks luis you said right ,i have to check how uploading takes place.. could you suggest some documnet ?? -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Sorry i think am taking lot of your time -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
[Mongrel] failed to allocate memory while downloading large files
hii , I uploaded 1 GB of file to filesystem,but now when i trying to download this 1 GB of file ,i am getting failed to allocate memory. am running mongrel as server in development mode. NoMemoryError (failed to allocate memory): C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `expand_cache_key' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/response.rb:118:in `etag=' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/response.rb:148:in `handle_conditional_get!' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/response.rb:138:in `prepare!' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:532:in `send_response' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:526:in `process_without_filters' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:606:in `process_without_session_ma gement_support' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/session_management.rb:134:in `process' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:392:in `process' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:183:in `handle_request' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:110:in `dispatch_unlocked' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:123:in `dispatch' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `synchronize' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `dispatch' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:132:in `dispatch_cgi' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:39:in `dispatch' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/bin/../lib/mongrel/rails.rb:76:in `process' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/bin/../lib/mongrel/rails.rb:74:in `synchronize' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/bin/../lib/mongrel/rails.rb:74:in `process' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:159:in `process_client' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:158:in `each' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:158:in `process_client' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:285:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:285:in `initialize' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:285:in `new' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:285:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:268:in `initialize' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:268:in `new' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel.rb:268:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel/configurator.rb:282:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel/configurator.rb:281:in `each' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel/configurator.rb:281:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/bin/mongrel_rails:128:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/lib/mongrel/command.rb:212:in `run' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/mongrel-1.1.2-x86-mswin32/bin/mongrel_rails:281 C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:142:in `load_without_new_cons nt_marking' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/dependencies.rb:142:in `load'
Re: [Mongrel] failed to allocate memory while downloading large files
On Mon, Sep 13, 2010 at 7:33 AM, Amit Tomar li...@ruby-forum.com wrote: hii , I uploaded 1 GB of file to filesystem,but now when i trying to download this 1 GB of file ,i am getting failed to allocate memory. am running mongrel as server in development mode. NoMemoryError (failed to allocate memory): C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `' C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `expand_cache_key' This has nothing to do with Mongrel. From the backtrace it seems you're trying to cache all the file contents in memory. What is the code you're using to serve the file? Mongrel is quite capable to serve any file from public folder without any Rails code interfering it. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exupéry ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
Re: [Mongrel] failed to allocate memory while downloading large files
Luis Lavena wrote: On Mon, Sep 13, 2010 at 7:33 AM, Amit Tomar li...@ruby-forum.com wrote: hii , ? ? ?I uploaded 1 GB of file to filesystem,but now when i trying to download this 1 GB of file ,i am getting failed to allocate memory. am running mongrel as server in development mode. NoMemoryError (failed to allocate memory): ?C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `' ?C:/InstantRails-2.0-win/ruby/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/cache.rb:60:in `expand_cache_key' This has nothing to do with Mongrel. From the backtrace it seems you're trying to cache all the file contents in memory. What is the code you're using to serve the file? Mongrel is quite capable to serve any file from public folder without any Rails code interfering it. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup鲹 Luis ,this is how am downloading my files send_data(@data,:filename = @containerformat.name+extension, :disposition = 'attachment') but i am getting error failed to allocate memory for large files ,for small files it's working fine and one thing more i observed while uploading large files , whenever i upload data more than 2gb from IE OR from mozila ,both restarts but for google chorme its working fine??? -- Posted via http://www.ruby-forum.com/. ___ Mongrel-users mailing list Mongrel-users@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users
