This is an automated email from the ASF dual-hosted git repository.

snagel pushed a commit to branch master
in repository

The following commit(s) were added to refs/heads/master by this push:
     new 010c2fc  NUTCH-2691: Improve logging from scoring-depth plugin
     new cb580f0  Merge pull request #434 from YossiTamari/patch-3
010c2fc is described below

commit 010c2fc8035525545812ae8acfbeeda1a8bbb96b
Author: YossiTamari <>
AuthorDate: Tue Jan 22 18:03:45 2019 +0200

    NUTCH-2691: Improve logging from scoring-depth plugin
    Exit distributeScoreToOutlinks immediately if there are no outlinks. This 
is a very small performance improvement, but more importantly it prevents the 
plugin from emitting a "Missing depth, removing all outlinks from url" warn 
message for every page that failed parsing.
 .../src/java/org/apache/nutch/scoring/depth/    | 3 +++
 1 file changed, 3 insertions(+)

diff --git 
index 07e0e3f..c016030 100644
@@ -73,6 +73,9 @@ public class DepthScoringFilter extends Configured implements 
ScoringFilter {
   public CrawlDatum distributeScoreToOutlinks(Text fromUrl,
       ParseData parseData, Collection<Entry<Text, CrawlDatum>> targets,
       CrawlDatum adjust, int allCount) throws ScoringFilterException {
+    if (targets.isEmpty()) {
+      return adjust;
+    }
     String depthString = parseData.getMeta(DEPTH_KEY);
     if (depthString == null) {
       LOG.warn("Missing depth, removing all outlinks from url " + fromUrl);

Reply via email to