[
https://issues.apache.org/jira/browse/NUTCH-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107943#comment-14107943
]
Sebastian Nagel commented on NUTCH-1308:
----------------------------------------
Hi Lewis, of course I meant that (2) is fixed by NUTCH-1603. Regarding (1):
NUTCH-965 was because some parsers hang-up with truncated documents. If all
parsers would know by themselves whether they can parse truncated pages or may
possibly fail, we would not need "parser.skip.truncated". So, parse-zip is
right in this point. We should keep the extra check, also for the case that
parser.skip.truncated == false.
Point (3): add main class (see patch)?
> Unnecessary truncate content configuration, and logging in
> parse-zip/ZipParser
> --------------------------------------------------------------------------------
>
> Key: NUTCH-1308
> URL: https://issues.apache.org/jira/browse/NUTCH-1308
> Project: Nutch
> Issue Type: Bug
> Components: parser
> Affects Versions: 1.4, nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Fix For: 2.4
>
> Attachments: NUTCH-1308-ZipParser-main-trunk.patch
>
>
> Two issues here...
> 1) Recently ferdy committed NUTCH-965 which skips parsing of truncated
> documents. Parse zip has it's own implementation for the same when it should
> really draw on the aforementioned implementation.
> 2) If (in the offending piece of code mentioned above) truncation occurs, we
> get an incorrect log message the "Parser can't handle incomplete pdf
> files"!!! This is incorrect, shouldn't be there, and should be removed.
> {code}
> 72 if (contentLen != null && contentInBytes.length != len) {
> 73 return new ParseStatus(ParseStatus.FAILED,
> 74 ParseStatus.FAILED_TRUNCATED, "Content truncated at "
> 75 + contentInBytes.length
> 76 + " bytes. Parser can't handle incomplete pdf file.")
> 77 .getEmptyParseResult(content.getUrl(), getConf());
> 78 }
> {code}
> For clarity, the issue is present in both Nutchgora branch[1] and Nutch
> trunk[2]
> [1]
> https://svn.apache.org/viewvc/nutch/branches/nutchgora/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip/ZipParser.java?diff_format=h&view=markup
> [2]
> https://svn.apache.org/viewvc/nutch/trunk/src/plugin/parse-zip/src/java/org/apache/nutch/parse/zip/ZipParser.java?diff_format=h&view=markup
> [2]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
