It wasn't necessary to deal with tools like tcpdump etc. Adding the following to the logging.xml did the trick: <Logger name="org.apache.http.wire" level="debug" additivity="false"> <AppenderRef ref="MyFile" /> </Logger>
So now I know what's going on. Bad credentials: DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "HTTP/1.1 401 Unauthorized[\r][\n]" Strange, because connection is working according to the Solr Output Connector. I'll double-check whether the Solr server has another password for index writing (path "/solr/uio/update/extract"). Or maybe we have an encoding issue with the password since it's long and contains special characters. --8<-- DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "</div><!-- container --> [\n]" DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "</body> [\n]" DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "</html>[\n]" DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "[\n]" DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "2f[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "******************[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "0[\r][\n]" DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "HTTP/1.1 401 Unauthorized[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Date: Thu, 27 Dec 2018 10:18:41 GMT[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "WWW-Authenticate: Basic realm="Solr"[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Content-Length: 381[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Keep-Alive: timeout=10, max=100[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Connection: Keep-Alive[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Content-Type: text/html; charset=iso-8859-1[\r][\n]" DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "[\r][\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "<html><head>[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "<title>401 Unauthorized</title>[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "</head><body>[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "<h1>Unauthorized</h1>[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "<p>This server could not verify that you[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "are authorized to access the document[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "requested. Either you supplied the wrong[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "credentials (e.g., bad password), or your[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "browser doesn't understand how to supply[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "the credentials required.</p>[\n]" DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "</body></html>[\n]" WARN 2018-12-27T11:18:41,599 (Worker thread '48') - IO exception during indexing https://www.journals.uio.no/index.php/Dialogia: null org.apache.http.client.ClientProtocolException Erlend On 21/12/2018 16:39, Karl Wright wrote: > I'll have a look as time permits, but it won't be for a couple of weeks. > Usually it's best to debug with http wire debugging, not packet > captures. I'm not an HTTP expert so it may be better to bring this up > to the HttpComponents/HttpClient list, not ManifoldCF. > > Karl > > > On Fri, Dec 21, 2018 at 8:54 AM Erlend Garåsen <[email protected] > <mailto:[email protected]>> wrote: > > > I tried to configure preemptive authentication by modifying the > HttpPoster class, but I still get the same errors. Then I ran the > following command in order to analyze the traffic: > tcpdump -w /tmp/chatter.dmp -s 0 -i ens192 -X host > solr-test01.uio.no <http://solr-test01.uio.no> > > I'm not an expert in reading such outputs, but I can send you the dump > file, Karl. I don't want to let the file be available for others, even > though it's from our test environment. > > Erlend > > On 13/12/2018 15:09, Karl Wright wrote: > > Ok, thanks, I misunderstood where the SSL error was coming from. The > > Solr connection is what is complaining. Do you see "Connection > working" > > for your output connection? Please forgive me if you already answered > > this; I didn't note it in your response. If you see that, then the > > connector was able to talk to your Solr "ping" handler, which > would mean > > that the SSL configuration is right but there's something else > about the > > connection that we would have to figure out and deal with. > > > > Karl > > > > > > On Thu, Dec 13, 2018 at 8:58 AM Erlend Garåsen > <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> > wrote: > > > > On 13/12/2018 14:26, Karl Wright wrote: > > > This is SSL. Did you add the server's cert to the web > connector's > > > keystore? Or, if not, add a "trust all" rule? > > > > Thanks for the reply, Karl. > > > > Yes, the root certificate was added both in the repository > connection > > and for the Solr connector. > > > > I checked the "trust all" option and restarted the job. Same > problem. > > > > BTW, I get a lot of 200s, so MCF is able to fetch the URLs. > > > > I have attached a screenshot of what I can see in the simple > history > > report. > > > > Erlend > > >
