Does anyone know if there's a maximum number of open connections allowed
when using hpricot/net::http?
I have a bunch of xml feeds I need to read via hpricot and write to disk.
Whenever I hit it too hard/fast, or when I try to open several files in
hpricot in a loop, my app dies.
If I do something like...
arr = [url1, url2, url2]
arr.each do|d|
doc = Hpricot(open(d))
doc.search(blah...)
File.open("#{RAILS_ROOT}/xml/#{file_name}.xml", 'w') {|f| f.write doc }
end
I get:
OpenURI::HTTPError: 500 Internal Server Error from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:277:in
`open_http'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:616:in
`buffer_open'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:164:in
`open_loop'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:162:in
`catch'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:162:in
`open_loop'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:132:in
`open_uri'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:518:in
`open'
from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/open-uri.rb:30:in
`open'
from (irb):11
from (irb):11:in `each'
from (irb):11
Am I missing something here, or is there a better way of doing this? I'm on
Ruby 1.8.7 and hpricot 0.8.2
Thanks
Robbie
--
You received this message because you are subscribed to the Google Groups "Ruby
or Rails Oceania" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/rails-oceania?hl=en.
