[
https://issues.apache.org/jira/browse/NUTCH-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218222#comment-16218222
]
ASF GitHub Bot commented on NUTCH-2450:
---------------------------------------
sebastian-nagel commented on a change in pull request #235: Fix for NUTCH-2450
by Kenneth McFarland
URL: https://github.com/apache/nutch/pull/235#discussion_r146774541
##########
File path: src/java/org/apache/nutch/parse/ParseOutputFormat.java
##########
@@ -379,15 +378,16 @@ public static String filterNormalize(String fromUrl,
String toUrl,
if (ignoreInternalLinks) {
if ("bydomain".equalsIgnoreCase(ignoreExternalLinksMode)) {
String toDomain = URLUtil.getDomainName(targetURL).toLowerCase();
- //FIXME: toDomain will never be null, correct?
if (toDomain == null || toDomain.equals(origin)) {
return null; // skip it
}
} else {
String toHost = targetURL.getHost().toLowerCase();
- //FIXME: toDomain will never be null, correct?
if (toHost == null || toHost.equals(origin)) {
- return null; // skip it
+ if (exemptionFilters == null // check if it is exempted?
Review comment:
The [exemption
filter|https://builds.apache.org/job/nutch-trunk/javadoc/org/apache/nutch/urlfilter/ignoreexempt/package-summary.html]
is to define exemption for otherwise skipped external links. Should not be in
the branch which handles internal links, otherwise we would have to redefine
the exemption filter interface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Remove FixMe in ParseOutputFormat
> ---------------------------------
>
> Key: NUTCH-2450
> URL: https://issues.apache.org/jira/browse/NUTCH-2450
> Project: Nutch
> Issue Type: Bug
> Environment: master branch
> Reporter: Kenneth McFarland
> Assignee: Kenneth McFarland
> Priority: Minor
>
> ParseOutputFormat contains a few FixMe's that I've looked at. If a valid url
> is created, it will always return valid results. There is a spot in the code
> where the try catch is already done, so the predicate is satisfied and there
> is no need to keep checking it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
