[
https://issues.apache.org/jira/browse/NUTCH-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14576720#comment-14576720
]
Sebastian Nagel commented on NUTCH-2035:
----------------------------------------
Thanks for benchmarking: since we both got roughly comparable results with the
same order: Let's change the file type exclusion pattern to {{-(?i)...}}.
Regarding the host pattern: we should also lowercase the pattern. But we do not
need necessarily the {{(?i)}} because URL normalizers are run before URL
filters and urlnormalizer-basic which is used per default converts the host
name to lowercase.
> Regex filter using case sensitive rules.
> ----------------------------------------
>
> Key: NUTCH-2035
> URL: https://issues.apache.org/jira/browse/NUTCH-2035
> Project: Nutch
> Issue Type: Improvement
> Components: plugin
> Affects Versions: 1.10
> Reporter: Luis Lopez
> Priority: Minor
> Labels: filters, regex, regex-urlfilter
> Fix For: 1.11
>
> Attachments: regex-urlfilter.txt
>
>
> Regex expressions are computationally expensive and having “EXE|exe|JPG|jpg”
> etc etc..... adds up if we use complex rules.
> Regex filter should use case insensitive rules to make the rules more
> readable and improve performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
