Author: dogacan
Date: Wed Jun 27 05:46:05 2007
New Revision: 551147
URL: http://svn.apache.org/viewvc?view=rev&rev=551147
Log:
NUTCH-498 - Use Combiner in LinkDb to increase speed of linkdb generation.
Contributed by Espen Amble Kolstad.
Modified:
lucene/nutch/trunk/CHANGES.txt
lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java
Modified: lucene/nutch/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/lucene/nutch/trunk/CHANGES.txt?view=diff&rev=551147&r1=551146&r2=551147
==============================================================================
--- lucene/nutch/trunk/CHANGES.txt (original)
+++ lucene/nutch/trunk/CHANGES.txt Wed Jun 27 05:46:05 2007
@@ -72,6 +72,9 @@
23. NUTCH-499 - Refactor LinkDb and LinkDbMerger to reuse code. (dogacan)
+24. NUTCH-498 - Use Combiner in LinkDb to increase speed of linkdb generation.
+ (Espen Amble Kolstad via dogacan)
+
Release 0.9 - 2007-04-02
1. Changed log4j confiquration to log to stdout on commandline
Modified: lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java
URL:
http://svn.apache.org/viewvc/lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java?view=diff&rev=551147&r1=551146&r2=551147
==============================================================================
--- lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java (original)
+++ lucene/nutch/trunk/src/java/org/apache/nutch/crawl/LinkDb.java Wed Jun 27
05:46:05 2007
@@ -215,6 +215,7 @@
job.setInputFormat(SequenceFileInputFormat.class);
job.setMapperClass(LinkDb.class);
+ job.setCombinerClass(LinkDbMerger.class);
// if we don't run the mergeJob, perform normalization/filtering now
if (normalize || filter) {
try {
