[
https://issues.apache.org/jira/browse/NUTCH-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754860#comment-16754860
]
Hudson commented on NUTCH-2689:
-------------------------------
FAILURE: Integrated in Jenkins build Nutch-trunk #3608 (See
[https://builds.apache.org/job/Nutch-trunk/3608/])
NUTCH-2689 Speed up urlfilter-regex and urlfilter-automaton - do not (snagel:
[https://github.com/apache/nutch/commit/f87b19b0ee8a01c5f54f5ed4b6b159169705682f])
* (edit) conf/regex-urlfilter.txt.template
* (edit)
src/plugin/lib-regex-filter/src/java/org/apache/nutch/urlfilter/api/RegexURLFilterBase.java
* (edit) src/plugin/urlfilter-regex/sample/Benchmarks.rules
* (edit) src/plugin/urlfilter-regex/sample/WholeWebCrawling.rules
* (edit) src/plugin/urlfilter-regex/sample/IntranetCrawling.rules
> Speed up urlfilter-regex and urlfilter-automaton
> ------------------------------------------------
>
> Key: NUTCH-2689
> URL: https://issues.apache.org/jira/browse/NUTCH-2689
> Project: Nutch
> Issue Type: Improvement
> Components: plugin
> Affects Versions: 1.15
> Reporter: Sebastian Nagel
> Assignee: Sebastian Nagel
> Priority: Minor
> Fix For: 1.16
>
>
> The unit tests of urlfilter-regex and urlfilter-automaton include a
> benchmark. After playing and benchmarking modifications the following changes
> seem to significantly improve the performance:
> - do not extract host and domain name from the URL if not needed (no
> host/domain-specific rules used, cf. NUTCH-1838)
> - use non-capturing groups if possible
> - use {{(?i)}} to make the patterns case insensitive and remove uppercase
> variants
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
