I've got a spider and I'm wondering how to correctly deal with basic authentication.
Are there response methods that will return the type (Basic or Digest) and realm of a 401 response? I couldn't find any in HTTP::Headers or HTTP::Response. I'm spidering and I'm not clear on is how to supply credentials automatically. That is, say http://foo.com/path/bar.html generates a 401. I query for the username and password, and then resubmit (I'm using simple_request() so I can't use the get_basic_credentials callback). Now, I extract out links, and one link is to href="baz.html". It seems like the only correct thing to do is to request baz.html, and if I get back a 401 with the same realm automatically send the same credentials again. Seems like a "waste" of a request cycle -- but I'm not clear how to determine that I should send the credentials unless I'm asked. Thanks, -- Bill Moseley mailto:[EMAIL PROTECTED]
