Ok, I have wiresharked it, and I cannot obviously share the account data, but I am seeing the conversation, and it goes like this.
Nutch to IIS => GET HTTP/1.1 IIS to Nutch => HTTP/1.1 401 Unauthorized Nutch to IIS => ACK Nutch to IIS => NTLMSSP_NEGOTIATE IIS => Nutch => HTTP/1.1 401 Unauthorized , NTLMSSP_CHALLENGE (text/html) Nutch to IIS => ACK Nutch to IIS => GET HTTP1.1 , NTLM_AUTH, User: DOAMAIN\USER IIS => Nutch => HTTP/1.1 401 Unauthorized Nutch to IIS => ACK Nutch to IIS => FIN,ACK IIS => Nutch => FIN/ACK Nutch to IIS => ACK I have located IIS Event logs, and they look a little weird to me and they are indicating a problem with login. So, either the data being delivered in the NTLM_AUTH by Nutch is bad, or the IIS is not allowing the auth in it's conversations with the AD, or IIS is not really supporting NTLM (working on validating this) This works: curl --anyauth --user <domainuser>:<password> http://iis75.intranet.org This fails: wget -c --http-user=' <domain>\<domainuser> --http-password=<password> http://iis75.intranet.org -dv [root@localhost nutch]# wget -c --http-user='coacd\bellb' --http-password=<password > http://iis75.intranet.org -dv Setting --verbose (verbose) to 1 DEBUG output created by Wget 1.14 on linux-gnu. URI encoding = ‘UTF-8’ --2016-11-04 09:43:00-- http://iis75.intranet.org Host ‘http://iis75.intranet.orghas not issued a general basic challenge. Resolving iis75.intranet.org (iis75.intranet.org)... 10.1.x.x Caching iis75.intranet.org => 10.1.x.x Connecting to iis75.intranet.org (iis75.intranet.org)|10.1.x.x|:80... connected. Created socket 3. Releasing 0x0000000001681c60 (new refcount 1). ---request begin--- GET / HTTP/1.1 Range: bytes=54599- User-Agent: Wget/1.14 (linux-gnu) Accept: */* Host: iis75.intranet.org Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/7.5 WWW-Authenticate: NTLM WWW-Authenticate: Basic realm="iis75.intranet.org" X-Powered-By: ASP.NET Date: Fri, 04 Nov 2016 14:43:00 GMT Content-Length: 0 ---response end--- 401 Unauthorized Registered socket 3 for persistent reuse. ] done. Empty NTLM message, starting transaction. Creating a type-1 NTLM message. Reusing existing connection to iis75.intranet.org:80. Reusing fd 3. ---request begin--- GET / HTTP/1.1 Range: bytes=54599- User-Agent: Wget/1.14 (linux-gnu) Accept: */* Host: iis75.intranet.org Connection: Keep-Alive Authorization: NTLM TlRMTVNTUAABAAAAAgIAAAAAAAAgAAAAAAAAACAAAAA= ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Content-Type: text/html; charset=us-ascii Server: Microsoft-HTTPAPI/2.0 WWW-Authenticate: NTLM <ntlm hash> Date: Fri, 04 Nov 2016 14:43:00 GMT Content-Length: 341 ---response end--- 401 Unauthorized Skipping 341 bytes of body: [<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd";> <HTML><HEAD><TITLE>Not Authorized</TITLE> <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD> <BODY><h2>Not Authorized</h2> <hr><p>HTTP Error 401. The requested resource requires user authentication.</p> </BODY></HTML> ] done. Received a type-2 NTLM message. Creating a type-3 NTLM message. Reusing existing connection to iis75.intranet.org:80. Reusing fd 3. ---request begin--- GET / HTTP/1.1 Range: bytes=54599- User-Agent: Wget/1.14 (linux-gnu) Accept: */* Host: iis75.intranet.org Connection: Keep-Alive Authorization: NTLM <ntlm hash> ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Cache-Control: no-cache, no-store, must-revalidate, max-age=0 Pragma: no-cache Transfer-Encoding: chunked Content-Type: text/html;charset=UTF-8 Expires: Thu, 01 Jan 1970 00:00:00 GMT Server: Microsoft-IIS/7.5 Set-Cookie: JSESSIONID=B062CF384F4ED170B59C2684DC65E40F.cfusion; Path=/; HttpOnly Set-Cookie: CFID=353750; Expires=Sun, 28-Oct-2046 14:43:01 GMT; Path=/; HttpOnly Set-Cookie: CFTOKEN=ab6d17259109ce88-DD15E3C9-91CC-22A2-6666A3D7A0A2FBC3; Expires=Sun, 28-Oct-2046 14:43:01 GMT; Path=/; HttpOnly Persistent-Auth: true X-Powered-By: ASP.NET Date: Fri, 04 Nov 2016 14:43:05 GMT ---response end--- 200 OK Stored cookie iis75.intranet.org -1 (ANY) / <session> <insecure> [expiry none] JSESSIONID B062CF384F4ED170B59C2684DC65E40F.cfusion Stored cookie iis75.intranet.org -1 (ANY) / <permanent> <insecure> [expiry 2046-10-28 09:43:01] CFID 353750 Stored cookie iis75.intranet.org -1 (ANY) / <permanent> <insecure> [expiry 2046-10-28 09:43:01] CFTOKEN ab6d17259109ce88-DD15E3C9-91CC-22A2-6666A3D7A0A2FBC3 URI content encoding = ‘UTF-8’ Length: unspecified [text/html] Saving to: ‘index.html’ [ <=> ] 0 --.-K/s in 0s Disabling further reuse of socket 3. Closed fd 3 -----Original Message----- From: Furkan KAMACI [mailto:[email protected]] Sent: Wednesday, November 02, 2016 3:53 PM To: [email protected] Subject: Re: Nutch 1.12 NTLM authentication IIS 7.5 Intranet Hi Bob, You may be right. Could you share the sniffed data. Do not forget to check headers when you analyse it. Kind Regards, Furkan KAMACI On Wed, Nov 2, 2016 at 10:23 PM, Bell, Bob <[email protected]> wrote: > This is my httpclient-auth.xml where domainuser is my ad account, > domainpassword is my password, and the realm is populated with our ad > domain. I have used host="ip of nutch box" > and host="website trying to crawl" I am not sure what you are trying to > say. I have used every > combination I can think of, and it just doesn't seem to work. > > <auth-configuration> > <credentials username="domainuser" password="domainpassword"> > <default host="ip of nutch box" scheme="ntlm" port="80" > realm="domain"/> > </credentials> > </auth-configuration> > > Is this not the proper way to setup the ntlm credentials above ? > > I have tried it with local accounts on the server being crawled, no luck > there either. I have used fully qualified domain names for > the ad domain, like domain.org etc.. Are the error logs pointing to > anything at all ?? > > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodBase - enter > HttpMethodBase.processResponseHeaders(HttpState, HttpConnection) > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodBase - enter > HttpMethodBase.processCookieHeaders(Header[], HttpState, > HttpConnection) > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodBase - enter > HttpMethodBase.readResponseBody(HttpState, HttpConnection) > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodBase - enter > HttpMethodBase.readResponseBody(HttpConnection) > 2016-11-02 14:41:12,059 TRACE httpclient.HttpConnection - enter > HttpConnection.getResponseInputStream() > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodBase - enter > HttpMethodBase.canResponseHaveBody(int) > > 2016-11-02 14:41:12,059 DEBUG httpclient.HttpMethodDirector - > Authorization required > 2016-11-02 14:41:12,059 TRACE httpclient.HttpMethodDirector - enter > HttpMethodBase.processAuthenticationResponse(HttpState, > HttpConnection) > 2016-11-02 14:41:12,059 DEBUG auth.AuthChallengeProcessor - Using > authentication scheme: ntlm > 2016-11-02 14:41:12,060 DEBUG auth.AuthChallengeProcessor - > Authorization challenge processed > 2016-11-02 14:41:12,060 DEBUG httpclient.HttpMethodDirector - > Authentication scope: NTLM <any realm>@iis75.intranet.org:80 > 2016-11-02 14:41:12,060 DEBUG httpclient.HttpMethodDirector - > Credentials required > 2016-11-02 14:41:12,060 DEBUG httpclient.HttpMethodDirector - > Credentials provider not available > 2016-11-02 14:41:12,060 INFO httpclient.HttpMethodDirector - Failure > authenticating with NTLM <any realm>@iis75.intranet.org:80 > > These previous entries bother me the most. What does "Credentials > provider not available" mean ? Then I see Failure authenticating with > NTLM, which a real failure, would cause issues on my IIS75, and lock > the accounts out ? Baffling, this is why I say I do not think it > really is failing, it just think it failed based on interpretation of > what > it is getting back from the iis server ? Im stumped. > > 2016-11-02 14:41:12,062 DEBUG httpclient.HttpMethodBase - Resorting to > protocol version default close connection policy > 2016-11-02 14:41:12,062 DEBUG httpclient.HttpMethodBase - Should NOT > close connection, using HTTP/1.1 > 2016-11-02 14:41:12,062 TRACE httpclient.HttpConnection - enter > HttpConnection.isResponseAvailable() > 2016-11-02 14:41:12,062 TRACE httpclient.HttpConnection - enter > HttpConnection.releaseConnection() > 2016-11-02 14:41:12,062 DEBUG httpclient.HttpConnection - Releasing > connection back to connection manager. > 2016-11-02 14:41:12,062 TRACE > httpclient.MultiThreadedHttpConnectionManager > - enter HttpConnectionManager.releaseConnection(HttpConnection) > 2016-11-02 14:41:12,062 DEBUG > httpclient.MultiThreadedHttpConnectionManager > - Freeing connection, hostConfig=HostConfiguration[host= > http://iis75.intranet.org] > 2016-11-02 14:41:12,062 TRACE > httpclient.MultiThreadedHttpConnectionManager > - enter HttpConnectionManager.ConnectionPool.getHostPool( > HostConfiguration) > 2016-11-02 14:41:12,062 DEBUG util.IdleConnectionHandler - Adding > connection at: 1478115672062 > 2016-11-02 14:41:12,062 DEBUG > httpclient.MultiThreadedHttpConnectionManager > - Notifying no-one, there are no waiting threads > 2016-11-02 14:41:12,062 TRACE httpclient.Http - url: > http://iis75.intranet.org; status code: 401; bytes received: 0; > Content-Length: 0 > 2016-11-02 14:41:12,367 DEBUG util.ObjectCache - No object cache found > for > conf=Configuration: core-default.xml, core-site.xml, > nutch-default.xml, nutch-site.xml, instantiating a new object cache > 2016-11-02 14:41:12,493 TRACE httpclient.Http - 401 Authentication > Required > > I see this same issue with other Nutch users when I google it, and > people point back to the limited documented settings. > I know ntlm on the server works, as 300+ plus users authenticate to it > every day. I am not seeing any > security logs on the iis75 server indicating bad login attempts, and I > would get locked out if there was. > I must assume, it is not really attempting to authenticate, I am about > to put a sniffer on it, look at the actual traffic. > > Thanks, > Bob > > -----Original Message----- > From: Furkan KAMACI [mailto:[email protected]] > Sent: Wednesday, November 02, 2016 3:00 PM > To: [email protected] > Subject: Re: Nutch 1.12 NTLM authentication IIS 7.5 Intranet > > Hi Bob, > > It's explained as: > > "Example:- > <credentials username="susam" password="masus"> > <default realm="sso"/> > <authscope host="192.168.101.33" port="80" realm="login"/> > <authscope host="example" port="8080" realm="blogs"/> > <authscope host="example" port="8080" realm="wiki"/> > <authscope host="example" port="80" realm="quiz" scheme="NTLM"/> > </credentials> > <credentials username="admin" password="nimda"> > <authscope host="example" port="8080"/> > </credentials> > > In the above example, 'example:8080' server has pages with multiple > authentication realms. The first set of credentials would be used for > 'blogs' and 'wiki' authentication realms. The second set of > credentials would be used for all other realms. For 'login' realm of > '192.168.101.33', the first set of credentials would be used. For any > other realm of '192.168.101.33' authentication would not be done. For > the NTLM authentication required by 'example:80', the first set of > credentials would be used. For 'sso' realms of all other servers, the > first set of credentials would be used, since it is configured as > 'default'. > > NTLM does not use the notion of realms. The domain name may be > specified as the value for 'realm' attribute in case of NTLM." > > So, do you set realm? > > Kind Regards, > Furkan KAMACI > > On Wed, Nov 2, 2016 at 9:49 PM, Bell, Bob <[email protected]> > wrote: > > > Furkan, > > > > Same results. I tried domain\\user and domain\user, do I need > > to put a trace on the traffic and see what packets are being sent > > by nutch ? > > > > Thanks, > > Bob > > > > -----Original Message----- > > From: Bell, Bob [mailto:[email protected]] > > Sent: Wednesday, November 02, 2016 2:31 PM > > To: [email protected] > > Subject: RE: Nutch 1.12 NTLM authentication IIS 7.5 Intranet > > > > Yes, I will check that. I cranked up the logging and ran again, to > see > > if you might spot something odd. > > > > > > -----Original Message----- > > From: Furkan KAMACI [mailto:[email protected]] > > Sent: Wednesday, November 02, 2016 2:20 PM > > To: [email protected] > > Cc: Bell, Bob <[email protected]> > > Subject: Re: Nutch 1.12 NTLM authentication IIS 7.5 Intranet > > > > Hi Bob, > > > > Server may require that the domain as a part of username. For > > example, "domain\\user". Could you check that? > > > > Kind Regards, > > Furkan KAMACI > > > > On Wed, Nov 2, 2016 at 9:11 PM, Bell, Bob <[email protected]> > > wrote: > > > > > I have replaced <iis74.intranet> is just a string replacement for > > > our actual intranet name something like blah.intranet.org, and I > > > use the <> convention when I obscuring actual data. > > > > > > What might the log4js.properties entry for httpclient.Http ? I > > > see it is only at INFO level logging, but I do not know that > > > proper object path to set it up. > > > > > > Thanks, > > > Bob > > > > > > >Hi Bob, > > > > > > > >Do you write host as <iis75.intranet> or iis75.intranet ? > > > > > > > >Kind Regards, > > > >Furkan KAMACI > > > > > > -----Original Message----- > > > From: Bell, Bob > > > Sent: Wednesday, November 02, 2016 12:17 PM > > > To: '[email protected]' <[email protected]> > > > Cc: Bell, Bob <[email protected]> > > > Subject: Nutch 1.12 NTLM authentication IIS 7.5 Intranet > > > > > > I have been trying for more than a year to get NTLM to work with > > > IIS > 7.5 > > > without success. I was > > > happy to see the 1.12 recent release, and thought ok I will give > > > it shot again. I am almost to point where I do not believe it > > > works with ntlm, or it does not know how to handle the multiple 401's > > > that are returned, or I have some fundamental problem somewhere ? I > > > have tried everything I > > > could think of, and am at loss on how to solve this mystery. My > Nutch > > > server is a Centos 7 in a > > > Virtual Box. I am using the httpclient as indicated in the docs but > > > with no love. I can fetch with > > > anonymous, but I need ntlm to work. > > > > > > I am using plugin.includes = >protocol-httpclient > > > > > > nutch-site.xml: > > > <property> > > > <name>http.auth.file</name> > > > <value>httpclient-auth.xml</value> > > > <description>Authentication configuration file for > 'protocol-httpclient' > > > plugin. > > > </description> > > > </property> > > > > > > httpclient-auth.xml for local user: > > > <auth-configuration> > > > <credentials username="nutch" password="<somepassword>"> > > > <default scheme="basic" port="80"/> > > > </credentials> > > > </auth-configuration> > > > > > > Here is output with local user account on the server, one thing I > > > notice, is that I cannot force authentication to be anything other > > > than ntlm, even though I support ntlm, basic, and > > > digest. Notice the scheme was basic, > > > but it goes though ntlm regardless. > > > > > > [root@localhost nutch]# nutch parsechecker http://<iis75.intranet> > > > fetching: http://<iis75.intranet> > > > Whitelisted hosts: [<iis75.intranet>] http.proxy.host = null > > > http.proxy.port = 8080 http.proxy.exception.list = false > > > http.timeout = 36000 http.content.limit = 65536 http.agent = > > > APL-Nutch-Spider/Nutch-1.12 http.accept.language = > > > en-us,en-gb,en;q=0.7,*;q=0.3 http.accept = > > > text/html,application/xhtml+ > > > xml,application/xml;q=0.9,*/*;q=0.8 > > > Credentials - username: nutch; set as default for realm: ; scheme: > > > basic Pre-configured credentials with scope - host: > > > <iis75.intranet>; > > > port: 80; not found for url: http://<iis75.intranet> Authorization > > > required Supported authentication schemes in the order of preference: > > > [ntlm, digest, basic] ntlm authentication scheme selected Using > > authentication scheme: > > > ntlm Authorization challenge processed Authentication scope: NTLM > > > <any > > > realm>@<iis75.intranet>:80 Credentials required Credentials > > > realm>provider not > > > available No credentials available for NTLM <any > > > realm>@<iis75.intranet>:80 > > > url: http://<iis75.intranet>; status code: 401; bytes received: 0; > > > Content-Length: 0 > > > 401 Authentication Required > > > Fetch failed with protocol status: access_denied(17), lastModified=0: > > > Authentication required: http://<iis75.intranet> [root@localhost > > > nutch]# > > > > > > > > > httpclient-auth.xml for domain user: > > > <auth-configuration> > > > <credentials username="<domainuser>" password="<domainpassword> > > > <default host="<iis75.intranet>" scheme="ntlm" port="80" > > > realm="<domain>"/> > > > </credentials> > > > </auth-configuration> > > > > > > note: doesn’t matter what I put in the host, doesn’t seem to > > > change anything. > > > > > > [root@localhost nutch]# nutch parsechecker http://<iis75.intranet> > > > fetching: http://<iis75.intranet> > > > Whitelisted hosts: [<iis75.intranet>] http.proxy.host = null > > > http.proxy.port = 8080 http.proxy.exception.list = false > > > http.timeout = 36000 http.content.limit = 65536 http.agent = > > > APL-Nutch-Spider/Nutch-1.12 http.accept.language = > > > en-us,en-gb,en;q=0.7,*;q=0.3 http.accept = > > > text/html,application/xhtml+ > > > xml,application/xml;q=0.9,*/*;q=0.8 > > > Credentials - username: <domainuser>"; set as default for realm: > > > =<domain>; scheme: ntlm Pre-configured credentials with scope - host: > > > <iis75.intranet>; port: 80; not found for url: > > > http://<iis75.intranet> Authorization required Supported > > > authentication schemes in the order of > > > preference: [ntlm, digest, basic] ntlm authentication scheme > > > selected Using authentication scheme: ntlm Authorization challenge > > > processed Authentication scope: NTLM <any > > > realm>@<iis75.intranet>:80 Retry authentication Authenticating > > > with NTLM <any > > > realm>@<iis75.intranet>:80 enter > > > realm>NTLMScheme.authenticate(Credentials, > > > HttpMethod) Authorization required Using authentication scheme: > > > ntlm Authorization challenge processed Authentication scope: NTLM > > > <any > > > realm>@<iis75.intranet>:80 Retry authentication Authenticating > > > realm>with > > > NTLM <any realm>@<iis75.intranet>:80 enter > > > NTLMScheme.authenticate(Credentials, HttpMethod) Authorization > > > required Using authentication scheme: ntlm Authorization challenge > > > processed Authentication scope: NTLM <any > > > realm>@<iis75.intranet>:80 Credentials required Credentials > > > provider not available Failure authenticating with NTLM <any > > > realm>@<iis75.intranet>:80 > > > url: http://<iis75.intranet>; status code: 401; bytes received: 0; > > > Content-Length: 0 > > > 401 Authentication Required > > > Fetch failed with protocol status: access_denied(17), lastModified=0: > > > Authentication required: http://<iis75.intranet> > > > > > > Last entry in Hadoop.log: > > > > > > 2016-11-02 12:08:49,568 INFO parse.ParserChecker - fetching: > > > http:// <iis75.intranet> > > > 2016-11-02 12:08:50,040 DEBUG util.ObjectCache - No object cache > > > found for > > > conf=Configuration: core-default.xml, core-site.xml, > > > nutch-default.xml, nutch-site.xml, instantiating a new object > > > cache > > > 2016-11-02 12:08:50,119 INFO protocol.RobotRulesParser - > > > Whitelisted > > > hosts: [<iis75.intranet>] > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - http.proxy.host = > > > null > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - http.proxy.port = > > > 8080 > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - > > > http.proxy.exception.list = false > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - http.timeout = > > > 36000 > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - http.content.limit > > > = > > > 65536 > > > 2016-11-02 12:08:50,119 INFO httpclient.Http - http.agent = > > > APL-Nutch-Spider/Nutch-1.12 ([email protected]) > > > 2016-11-02 12:08:50,120 INFO httpclient.Http - > > > http.accept.language = > > > en-us,en-gb,en;q=0.7,*;q=0.3 > > > 2016-11-02 12:08:50,120 INFO httpclient.Http - http.accept = > > > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > > > 2016-11-02 12:08:50,133 TRACE httpclient.Http - Credentials - username: > > > <domainuser>; set as default for realm: <domain>; scheme: ntlm > > > 2016-11-02 12:08:50,134 TRACE httpclient.Http - Pre-configured > > > credentials with scope - host: <iis75.intranet>; port: 80; not > > > found for url: http:// <iis75.intranet> > > > 2016-11-02 12:08:50,313 DEBUG httpclient.HttpMethodDirector - > > > Authorization required > > > 2016-11-02 12:08:50,320 DEBUG auth.AuthChallengeProcessor - > > > Supported authentication schemes in the order of preference: > > > [ntlm, digest, basic] > > > 2016-11-02 12:08:50,320 INFO auth.AuthChallengeProcessor - ntlm > > > authentication scheme selected > > > 2016-11-02 12:08:50,320 DEBUG auth.AuthChallengeProcessor - Using > > > authentication scheme: ntlm > > > 2016-11-02 12:08:50,320 DEBUG auth.AuthChallengeProcessor - > > > Authorization challenge processed > > > 2016-11-02 12:08:50,320 DEBUG httpclient.HttpMethodDirector - > > > Authentication scope: NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,320 DEBUG httpclient.HttpMethodDirector - > > > Retry authentication > > > 2016-11-02 12:08:50,321 DEBUG httpclient.HttpMethodDirector - > > > Authenticating with NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,321 TRACE auth.NTLMScheme - enter > > > NTLMScheme.authenticate(Credentials, HttpMethod) > > > 2016-11-02 12:08:50,351 DEBUG httpclient.HttpMethodDirector - > > > Authorization required > > > 2016-11-02 12:08:50,352 DEBUG auth.AuthChallengeProcessor - Using > > > authentication scheme: ntlm > > > 2016-11-02 12:08:50,352 DEBUG auth.AuthChallengeProcessor - > > > Authorization challenge processed > > > 2016-11-02 12:08:50,352 DEBUG httpclient.HttpMethodDirector - > > > Authentication scope: NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,352 DEBUG httpclient.HttpMethodDirector - > > > Retry authentication > > > 2016-11-02 12:08:50,352 DEBUG httpclient.HttpMethodDirector - > > > Authenticating with NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,352 TRACE auth.NTLMScheme - enter > > > NTLMScheme.authenticate(Credentials, HttpMethod) > > > 2016-11-02 12:08:50,393 DEBUG httpclient.HttpMethodDirector - > > > Authorization required > > > 2016-11-02 12:08:50,393 DEBUG auth.AuthChallengeProcessor - Using > > > authentication scheme: ntlm > > > 2016-11-02 12:08:50,393 DEBUG auth.AuthChallengeProcessor - > > > Authorization challenge processed > > > 2016-11-02 12:08:50,393 DEBUG httpclient.HttpMethodDirector - > > > Authentication scope: NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,393 DEBUG httpclient.HttpMethodDirector - > > > Credentials required > > > 2016-11-02 12:08:50,393 DEBUG httpclient.HttpMethodDirector - > > > Credentials provider not available > > > 2016-11-02 12:08:50,393 INFO httpclient.HttpMethodDirector - > > > Failure authenticating with NTLM <any realm>@<iis75.intranet>:80 > > > 2016-11-02 12:08:50,395 TRACE httpclient.Http - url: > > > http://<iis75.intranet>; status code: 401; bytes received: 0; > > > Content-Length: 0 > > > 2016-11-02 12:08:50,681 DEBUG util.ObjectCache - No object cache > > > found for > > > conf=Configuration: core-default.xml, core-site.xml, > > > nutch-default.xml, nutch-site.xml, instantiating a new object > > > cache > > > 2016-11-02 12:08:50,804 TRACE httpclient.Http - 401 Authentication > > > Required > > > > > > Any help is appreciated, as I am about to move on to another > > > spirder for solr. > > > > > > Thanks, > > > Bob > > > > > > > > >
