On Dec 6, 2011, at 12:17 PM, Javier Quarite wrote: > I grab some info from tha main page of the url (so it works) but when I enter > to its login page with user/pass and try to get some, it seems to get > information from other place (I'm not even sure from where)
Try all this out in a terminal with telnet or cURL -- see where you're actually going when you log in. You may be redirected in some subtle way. Also, a browser may throw a "basic authentication" dialog box when you're actually being challenged for digest authentication. :basic_authentication is not the same thing. I think your real solution here will be to abstract out the open() bit inside the Nokogiri::HTML() call. Look for a gem that accepts a URL and returns a text stream and offers a whole bunch of configuration options for authentication. I am certain there are at least a handful of them out there. By separating your concerns in this way, you'll end up with a more modular solution so that you can swap in different credentials for each site you're scraping. Walter -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
