[
https://issues.apache.org/jira/browse/NUTCH-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930477#comment-17930477
]
ASF GitHub Bot commented on NUTCH-3108:
---------------------------------------
lewismc commented on PR #849:
URL: https://github.com/apache/nutch/pull/849#issuecomment-2683539166
@maciejpuzianowski thanks for the PR
@tballison any comments on the tika dependency chain in Nutch. Should we be
looking to upgrade tika to 3.1.0? Would that address the slf4j issue.
Can you please link to the PR which introduced your shaded jar in place of
the official tika jar(s)?
Also, from memory, I believe that an upgrade to the core Hadoop libraries
may also impact this dependency tree.
> Fix SLF4J Class Loader Conflict in language-identifier
> ------------------------------------------------------
>
> Key: NUTCH-3108
> URL: https://issues.apache.org/jira/browse/NUTCH-3108
> Project: Nutch
> Issue Type: Bug
> Components: plugin
> Affects Versions: 1.20
> Reporter: Maciej Puzianowski
> Priority: Major
>
> When running Apache Nutch 1.20 on a distributed Hadoop cluster with the
> language-identifier plugin enabled, a class loader conflict occurs during the
> parse process. This results in the following error:
> {code:java}
> 2025-02-24 08:58:59,152 INFO mapreduce.Job: Task Id :
> attempt_1740061418437_0135_m_000001_0, Status : FAILED
> Error: loader constraint violation: when resolving method
> 'org.slf4j.ILoggerFactory
> org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader
> org.apache.nutch.plugin.PluginClassLoader @4c5228e7 of the current class,
> org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining
> class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for
> the type org/slf4j/ILoggerFactory used in the signature
> (org.slf4j.LoggerFactory is in unnamed module of loader
> org.apache.nutch.plugin.PluginClassLoader @4c5228e7, parent loader 'app';
> org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')
> {code}
> I have managed to resolve this issue by modifying following files:
> ivy.xml ->
> {code:xml}
> <dependency org="org.apache.tika" name="tika-langdetect-optimaize"
> rev="2.9.0" conf="*->default">
> <!-- exclusions of dependencies provided in Nutch core (ivy/ivy.xml) -->
> <exclude org="org.apache.tika" name="tika-core" />
> <exclude org="com.google.guava" name="guava" />
> <exclude org="org.slf4j" name="slf4j-api" />
> <!-- exclusions of dependencies provided in Nutch core (ivy/ivy.xml) -->
> </dependency>
> {code}
> and plugin.xml ->
> {code:xml}
> <library name="annotations-12.0.jar"/>
> <library name="checker-qual-3.33.0.jar"/>
> <library name="error_prone_annotations-2.18.0.jar"/>
> <library name="failureaccess-1.0.1.jar"/>
> <library name="j2objc-annotations-2.8.jar"/>
> <library name="jsonic-1.2.11.jar"/>
> <library name="jsr305-3.0.2.jar"/>
> <library name="language-detector-0.6.jar"/>
> <library
> name="listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar"/>
> <library name="tika-langdetect-optimaize-2.9.0.jar"/>
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
