[
https://issues.apache.org/jira/browse/NUTCH-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17451234#comment-17451234
]
ASF GitHub Bot commented on NUTCH-2891:
---------------------------------------
sebastian-nagel commented on pull request #700:
URL: https://github.com/apache/nutch/pull/700#issuecomment-982797518
Ok, the test crawl was successful. The only dubious errors which need a look
at are NoClassDefFoundError parsing files of MIME type
`application/x-7z-compressed`. E.g. reproducible by parsing one of the Tika
test documents:
```
bin/nutch parsechecker -Dplugin.includes='protocol-file|parse-tika'
file:/path/to/tika/repo/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pkg-module/src/test/resources/test-documents/test-documents.7z
```
Fixed by removing common-codec and commons-compress from the exclusions.
Although we have the same versions in core and parse-tika dependencies.
Obviously an issue with the plugin and core/parent class loaders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Upgrade to Tika 2.1
> -------------------
>
> Key: NUTCH-2891
> URL: https://issues.apache.org/jira/browse/NUTCH-2891
> Project: Nutch
> Issue Type: Improvement
> Components: parser, plugin
> Affects Versions: 1.18
> Reporter: Sebastian Nagel
> Assignee: Sebastian Nagel
> Priority: Major
> Fix For: 1.19
>
>
> There's already the second release of Tika 2
> ([2.1.0|https://tika.apache.org/2.1.0/index.html]). Following the [2.0
> release notes|https://archive.apache.org/dist/tika/2.0.0/CHANGES-2.0.0.txt]
> and the [migration
> guide|https://cwiki.apache.org/confluence/display/TIKA/Migrating+to+Tika+2.0.0]:
> * Tika 2 is more modular which should allow us to build a smaller parse-tika
> (66 MiB in the 1.18 binary package) by dropping rarely used parsers - but
> users should be able to include them if they build Nutch from the sources.
> * the language-identifier plugin needs to be upgraded as well (in addition to
> Nutch core and the parse-tika plugin). This would include or overlap with
> NUTCH-2449.
> * to avoid that the PDF parser times out we probably want to disable the OCR
> by default, or at least, provide the configuration snippet for this purpose
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
