Hello,
I was not able to find answer to this questions.
Can nutch crawler be configured to
1. crawl password protected site. i.e. Can I configure
crawler in such a way that it knows what user name and
password to enter on protected page before crawling
protected part of the site.
2. Site with java script agreement button. Can i
configure crawler to click on agreement/disclaimer
button before starting the crawl of the site which
requires agreement/disclaimer.
for example, I know that google appliance boxes can do
that.
Thank you
____________________________________________________________________________________
Sick sense of humor? Visit Yahoo! TV's
Comedy with an Edge to see what's on, when.
http://tv.yahoo.com/collections/222
