[ https://issues.apache.org/jira/browse/NUTCH-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1389: ---------------------------------------- Fix Version/s: 2.1 1.6 Aye this is a nice ticket Sebastian. There is work to be done on the Parserchecker within Nutchgora therefore I'll port this to Nutchgora as well when it is ready. > parsechecker and indexchecker to report truncated content > --------------------------------------------------------- > > Key: NUTCH-1389 > URL: https://issues.apache.org/jira/browse/NUTCH-1389 > Project: Nutch > Issue Type: Improvement > Components: indexer, parser > Affects Versions: nutchgora, 1.5 > Reporter: Sebastian Nagel > Priority: Minor > Fix For: 1.6, 2.1 > > > ParserChecker and IndexingFiltersChecker should report when a document is > truncated due to {http,file,ftp}.content.limit. > Truncated content may cause text and metadata extraction to fail for PDF and > other binary document formats. > A hint that truncation (and not a broken plugin) is the possible reason would > be useful. > See NUTCH-965 and {{ParseSegment.isTruncated(content)}}. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira