>From the logs, it looks like your server requires NTLM authentication. Could you please go through the "Need Help?" section of http://wiki.apache.org/nutch/HttpAuthenticationSchemes and provide all the information requested there?
Regards, Susam Pal On Wed, Dec 15, 2010 at 10:30 PM, Claudio Martella < [email protected]> wrote: > Hello list, > > I'm trying to crawl an intranet site which is behind authentication. The > webserver is behind Digest authentication. > My plugin.includes has the protocol-httpclient specified and I have > httpclient-auth.xml set like this: > > <auth-configuration> > <credentials username="user" password="password"> > <default scheme="digest"/> > </credentials> > </auth-configuration> > > I've also tried without specifying the scheme. Here's what comes out of > the httpclient logs: > > Supported authentication schemes in the order of preference: [ntlm, > digest, basic] > ntlm authentication scheme selected > Using authentication scheme: ntlm > Authorization challenge processed > Supported authentication schemes in the order of preference: [ntlm, > digest, basic] > ntlm authentication scheme selected > Using authentication scheme: ntlm > Authorization challenge processed > > Here's a like from hadoop.log > > 2010-12-15 17:51:29,853 INFO httpclient.HttpMethodDirector - No > credentials available for NTLM <any realm>@192.168.10.210:8090 > > I've also tried an <authscope host="192.168.10.210" port="8090" > scheme="digest"/> but nothings changes. > > Does anybody have an idea of what's going on? I'm using nutch 1.2 in > standalone mode. > > > Thanks > > -- > Claudio Martella > Digital Technologies > Unit Research & Development - Analyst > > TIS innovation park > Via Siemens 19 | Siemensstr. 19 > 39100 Bolzano | 39100 Bozen > Tel. +39 0471 068 123 > Fax +39 0471 068 129 > [email protected] http://www.tis.bz.it > > Short information regarding use of personal data. According to Section 13 > of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we > process your personal data in order to fulfil contractual and fiscal > obligations and also to send you information regarding our services and > events. Your personal data are processed with and without electronic means > and by respecting data subjects' rights, fundamental freedoms and dignity, > particularly with regard to confidentiality, personal identity and the right > to personal data protection. At any time and without formalities you can > write an e-mail to [email protected] in order to object the processing of > your personal data for the purpose of sending advertising materials and also > to exercise the right to access personal data and other rights referred to > in Section 7 of Decree 196/2003. The data controller is TIS Techno > Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the > complete information on the web site www.tis.bz.it. > > >
