[
https://issues.apache.org/jira/browse/NIFI-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17833335#comment-17833335
]
Joe Witt commented on NIFI-12996:
---------------------------------
First noted we pull in zstd-jni 1.5.5-6 whereas commons-compress expects
1.5.6-1. Though fixing that did not fix the problem.
Attaching a debugger to a running NiFi where this problem can be reproduced
showed that the 'CompressContent' class is tied to the proper nifi-standard-nar
which is correct and is indeed where zstd-jni lives. However looking at the
ZstdCompressorOutputStream class shows it is tied to the parent nar which is
'nifi-shared-services-nar' and that does NOT have the zstd-jni and we're.
parent first loading. This is why we're seeing the issue.
Solution:
(A) Remove commons-compress from the shared-services-nar (OR)
(B) Ensure optional dependencies of commons-compress are ALSO promoted to the
shared-services-nar with commons-compress
Also need to ensure these optional dependencies are well maintained.
Also need to review for any other such cases of promoted dependencies which
have optional deps like this as the same problem would occur.
> nifi-shared-bom construct broke classloading for optional dependencies of
> common-compress including zstd
> --------------------------------------------------------------------------------------------------------
>
> Key: NIFI-12996
> URL: https://issues.apache.org/jira/browse/NIFI-12996
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.24.0, 1.25.0, 2.0.0-M2
> Reporter: Joe Witt
> Assignee: Joe Witt
> Priority: Critical
> Fix For: 2.0.0-M3, 1.26.0
>
>
> In Apache Slack a user reported this was broken for them in NiFi 1.25.0 but
> used to work in 1.19.1.
> On apache 2.0 m2+latest I can reproduce this by simply trying to use it at
> all. First question is do we not have unit tests but anyway this needs to be
> evaluated as to why it isn't working.
> {noformat}
> 2024-04-02 13:30:12,514 ERROR [Timer-Driven Process Thread-3]
> o.a.n.p.standard.CompressContent
> CompressContent[id=a07f7ae5-018e-1000-c82c-2e2143f9c320] Processing halted:
> yielding [1 sec]
> java.lang.NoClassDefFoundError: com/github/luben/zstd/ZstdOutputStream
> at
> org.apache.commons.compress.compressors.zstandard.ZstdCompressorOutputStream.<init>(ZstdCompressorOutputStream.java:57)
> at
> org.apache.nifi.processors.standard.CompressContent$1.process(CompressContent.java:353)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3426)
> at
> org.apache.nifi.processors.standard.CompressContent.onTrigger(CompressContent.java:300)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: java.lang.ClassNotFoundException:
> com.github.luben.zstd.ZstdOutputStream
> at
> java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> ... 15 common frames omitted
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
