[
https://issues.apache.org/jira/browse/NUTCH-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16294209#comment-16294209
]
Hudson commented on NUTCH-2034:
-------------------------------
SUCCESS: Integrated in Jenkins build Nutch-trunk #3486 (See
[https://builds.apache.org/job/Nutch-trunk/3486/])
NUTCH-2034 CrawlDB update job to count documents in CrawlDb rejected by
(snagel:
[https://github.com/apache/nutch/commit/961c725aba2d6013a343dca66f595d6f28293a7b])
* (edit) src/java/org/apache/nutch/crawl/CrawlDb.java
* (edit) src/java/org/apache/nutch/crawl/CrawlDbFilter.java
> CrawlDB filtered documents counter.
> -----------------------------------
>
> Key: NUTCH-2034
> URL: https://issues.apache.org/jira/browse/NUTCH-2034
> Project: Nutch
> Issue Type: Improvement
> Components: crawldb
> Affects Versions: 1.10
> Reporter: Luis Lopez
> Assignee: Lewis John McGibbney
> Priority: Minor
> Labels: counters, crawldb, filter, info, regex
> Fix For: 1.14
>
>
> When we are doing big crawls we would like to know how many of the URLs are
> being discarded by the regex filters, this is only presented in the Inject
> class:
> Injector: Total number of urls rejected by filters: 0
> It will be nice to have a counter in the CrawlDB class so we know in every
> round how many were discarded by our filters:
> CrawlDb update: Total number of URLs filtered by regex filters: 31415
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
