[ https://issues.apache.org/jira/browse/NUTCH-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799038#comment-16799038 ]
Markus Jelsma commented on NUTCH-2703: -------------------------------------- This patch applies to the current Github master source, although it should probably also apply cleanly to the latest Nutch version. After patching source (patch -p1 < patchfile), Nutch must be rebuilt using ant. > parse-tika: Boilerpipe should not run for non-(X)HTML pages > ----------------------------------------------------------- > > Key: NUTCH-2703 > URL: https://issues.apache.org/jira/browse/NUTCH-2703 > Project: Nutch > Issue Type: Bug > Components: parser, plugin > Affects Versions: 1.15 > Reporter: Hany Shehata > Priority: Critical > Fix For: 1.16 > > Attachments: NUTCH-2703.patch > > > Boilerpipe is running for non-(X)html pages which is require more resources. > In my testing scenario, I've large PDFs in my websites and by enabling > Boilerpipe I have to assign 8500MB for JAVA Heap to finish the crawl job > without issues. > Disabling Boilerpipe allow me to minimize the JVM Heap to 500MB with no > issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)