Hello - http.timeout is in milliseconds. Apparently, those sites are not able to deliver a page in 7 seconds, for any reason. Increase to 30000 and it should work just fine for most sites. If you are blocked by a firewall, any timeout is never going to be high enough.
Regards, Markus -----Original message----- > From:suyashaoc <[email protected]> > Sent: Saturday 18th March 2017 8:51 > To: [email protected] > Subject: SocketTimeOutException is coming even after increasing http.timeout > > Hello Nutch-Users, > > I crawled 1000 urls setting http.timeout to 7000. But more than 200 urls are > throwing SocketTimeOut exception. > > I also tried increasing number of fetcher threads. That also did not worked. > > Please help how to fix. > > Regards, > Suyash > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/SocketTimeOutException-is-coming-even-after-increasing-http-timeout-tp4325750.html > Sent from the Nutch - User mailing list archive at Nabble.com. >
