Hello everyone,
i'm having a problem using Nutch to crawl a website that requires
authentication, https://www.romnetwerk.nl/.
I am using Nutch 1.11. Following the "start out as simple as possible"-advice,
my httpclient-auth.xml looks like this:
<credentials username=[username] password=[password]>
<default/>
</credentials>
I followed the debug-steps from
http://wiki.apache.org/nutch/HttpAuthenticationSchemes?highlight=%28%28HttpPostAuthentication%29%29
The first five steps are OK, up until the credentials showing up in the debug
logs, but the following line is NOT showing up: "auth.AuthChallengeProcessor -
basic authentication scheme selected" (or similar). The tutorial says that this
is a server side problem. Is there a way to make this work from my side?
Thanks in advance,
Vincent
