Author: jnioche
Date: Thu Jan 5 11:05:43 2012
New Revision: 1227553
URL: http://svn.apache.org/viewvc?rev=1227553&view=rev
Log:
NUTCH-1146 Prevent generation of _SUCCESS files in output
Modified:
nutch/trunk/CHANGES.txt
nutch/trunk/conf/nutch-default.xml
Modified: nutch/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/nutch/trunk/CHANGES.txt?rev=1227553&r1=1227552&r2=1227553&view=diff
==============================================================================
--- nutch/trunk/CHANGES.txt (original)
+++ nutch/trunk/CHANGES.txt Thu Jan 5 11:05:43 2012
@@ -1,5 +1,7 @@
Nutch Change Log
+* NUTCH-1146 Prevent generation of _SUCCESS files in output (jnioche)
+
* NUTCH-1232 Remove site field from index-basic (markus)
* NUTCH-1239 Webgraph should remove deleted pages from segment input (markus)
Modified: nutch/trunk/conf/nutch-default.xml
URL:
http://svn.apache.org/viewvc/nutch/trunk/conf/nutch-default.xml?rev=1227553&r1=1227552&r2=1227553&view=diff
==============================================================================
--- nutch/trunk/conf/nutch-default.xml (original)
+++ nutch/trunk/conf/nutch-default.xml Thu Jan 5 11:05:43 2012
@@ -1165,6 +1165,15 @@
</description>
</property>
+<property>
+ <name>mapreduce.fileoutputcommitter.marksuccessfuljobs</name>
+ <value>false</value>
+ <description>Hadoop >= 0.21 generates SUCCESS files in the output which can
crash
+ the readers. This should not be an issue once Nutch is ported to the new
MapReduce API
+ but for now this parameter should prevent such cases.
+ </description>
+</property>
+
<!-- solr index properties -->
<property>
