[
https://issues.apache.org/jira/browse/NUTCH-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581855#comment-17581855
]
ASF GitHub Bot commented on NUTCH-2930:
---------------------------------------
sebastian-nagel merged PR #736:
URL: https://github.com/apache/nutch/pull/736
> Protocol-okhttp: implement IP filter
> ------------------------------------
>
> Key: NUTCH-2930
> URL: https://issues.apache.org/jira/browse/NUTCH-2930
> Project: Nutch
> Issue Type: Improvement
> Components: plugin, protocol, security
> Affects Versions: 1.18
> Reporter: Sebastian Nagel
> Assignee: Sebastian Nagel
> Priority: Major
> Fix For: 1.19
>
>
> In order to avoid information leakage to a public search index or web
> archive, it should be possible to configure Nutch in a way that no content is
> fetched from localhost, loop-back addresses, private address spaces.
> NUTCH-2527 adds the configuration snippets to exclude URLs pointing to
> private addresses.
> However, filtering URLs isn't enough because a DNS entry of an arbitrary host
> name may point to a private IP address. Blocking must happen on the protocol
> level because the IP address is only know in the protocol implementation.
> I'll add an implementation for protocol-okhttp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
