On 6/22/2010 9:29 AM, Andrew Farnsworth wrote:
Don,
I have already tried using the cookie file from the browser and it
didn't work. I am talking about user/password, not https / ssl. I
get the warning from wget about not being able to verify the
certificate, but I use the flag "--no-check-certificate" to ignore
that error and it works fine.
According to the wget docs, I should use the
--save-cookies="cookieFile.txt" flag can call the login URL and then
use the --load-cookies="cookieFile.txt" flag and call the main URL to
crawl. That is where it just returns the "you need to login" page.
Andy
Two thoughts, firstly, presumably, this is web-based authentication as
opposed to HTTP based authentication? If not, you'll need to do things
differently.
Secondly, are you possibly crawling to a page that logs you out? On some
sites I have encountered, the front page will do this.
Rich
--
You received this message because you are subscribed to the Google Groups
"NLUG" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nlug-talk?hl=en
