[
https://issues.apache.org/jira/browse/NUTCH-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939430#comment-16939430
]
Sebastian Nagel edited comment on NUTCH-2457 at 9/27/19 1:12 PM:
-----------------------------------------------------------------
Actually, Nutch calls
{noformat}
CompositeParser compositeParser = (CompositeParser) tikaConfig.getParser();
Parser parser = compositeParser.getParsers().get(MediaType.parse(mimeType));
{noformat}
But it works for embedded documents only for a unit test, I've just added (see
PR). Running the parser checker, the embedded documents are not parsed:
{noformat}
> bin/nutch parsechecker -Dplugin.includes='protocol-file|parse-tika' -dumpText
> file:/.../test_recursive_embedded.docx
...
contentType:
application/vnd.openxmlformats-officedocument.wordprocessingml.document
...
embed_0
{noformat}
[[email protected]], is this caused by the way the parser is called? It
might be also because of parse-tika (as a Nutch plugin) holds the tika-parsers
jar in the plugin class loader while the tika-core jar is in the main class
loader (because it is required for MIME detection).
was (Author: wastl-nagel):
Actually, Nutch calls
{noformat}
CompositeParser compositeParser = (CompositeParser) tikaConfig.getParser();
Parser parser = compositeParser.getParsers().get(MediaType.parse(mimeType));
{noformat}
But it works for embedded documents only for a unit test, I've just added (see
PR). Running the parser checker, the embedded documents are not parsed:
{noformat}
> parsechecker -Dplugin.includes='protocol-file|parse-tika' -dumpText
> file:/.../test_recursive_embedded.docx
...
contentType:
application/vnd.openxmlformats-officedocument.wordprocessingml.document
...
embed_0
{noformat}
[[email protected]], is this caused by the way the parser is called? It
might be also because of parse-tika (as a Nutch plugin) holds the tika-parsers
jar in the plugin class loader while the tika-core jar is in the main class
loader (because it is required for MIME detection).
> Embedded documents likely not correctly parsed by Tika
> ------------------------------------------------------
>
> Key: NUTCH-2457
> URL: https://issues.apache.org/jira/browse/NUTCH-2457
> Project: Nutch
> Issue Type: Bug
> Affects Versions: 1.14
> Reporter: Tim Allison
> Priority: Major
> Fix For: 1.16
>
>
> While working on TIKA-2490, I think I found that Nutch's current method of
> requesting a mime-specific parser for each file will fail to parse embedded
> files, e.g.
> https://github.com/apache/tika/blob/master/tika-server/src/test/resources/test_recursive_embedded.docx
> The fix should be straightforward, and I'll submit a PR once I can get Nutch
> up and running in my dev environment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
