Author: cutting
Date: Tue Nov 22 10:55:26 2005
New Revision: 348212
URL: http://svn.apache.org/viewcvs?rev=348212&view=rev
Log:
Increase defaults for http.max.delays, since, with MapReduce's partitioning of
fetchlists, delays are more likely.
Modified:
lucene/nutch/branches/mapred/conf/crawl-tool.xml
lucene/nutch/branches/mapred/conf/nutch-default.xml
Modified: lucene/nutch/branches/mapred/conf/crawl-tool.xml
URL:
http://svn.apache.org/viewcvs/lucene/nutch/branches/mapred/conf/crawl-tool.xml?rev=348212&r1=348211&r2=348212&view=diff
==============================================================================
--- lucene/nutch/branches/mapred/conf/crawl-tool.xml (original)
+++ lucene/nutch/branches/mapred/conf/crawl-tool.xml Tue Nov 22 10:55:26 2005
@@ -33,7 +33,7 @@
<property>
<name>http.max.delays</name>
- <value>100</value>
+ <value>1000</value>
<description>The number of times a thread will delay when trying to
fetch a page. When using the crawl tool there are likely to be very
few different hosts, so we need to be willing to wait longer for
Modified: lucene/nutch/branches/mapred/conf/nutch-default.xml
URL:
http://svn.apache.org/viewcvs/lucene/nutch/branches/mapred/conf/nutch-default.xml?rev=348212&r1=348211&r2=348212&view=diff
==============================================================================
--- lucene/nutch/branches/mapred/conf/nutch-default.xml (original)
+++ lucene/nutch/branches/mapred/conf/nutch-default.xml Tue Nov 22 10:55:26 2005
@@ -69,7 +69,7 @@
<property>
<name>http.max.delays</name>
- <value>3</value>
+ <value>100</value>
<description>The number of times a thread will delay when trying to
fetch a page. Each time it finds that a host is busy, it will wait
fetcher.server.delay. After http.max.delays attepts, it will give
