[
https://issues.apache.org/jira/browse/NIFI-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471446#comment-16471446
]
Joseph Witt commented on NIFI-5184:
-----------------------------------
It is a known issue that once you load a native lib it is going to stay there
and you cannot reload that same library (by name anyway) anywhere else across
the JVM. This JVM behavior then imposes a limitation on how far our
classloader isolation can go - native libs can be a problem.
However, since you point out this happens during enable/disable of the
controller service I'm wondering if we have instance classloading in this
chain. If we do we might be able to stop doing that (not sure) and then it
would be fine since it would only load it once (unless something else already
loaded it). If something else also loaded that native library then one option
would be to exclude whatever else is doing that if we can since it should only
need to be in one place.
NiFi's classloader model does ensure there are a range of important and shared
across all classloader classes. We of course try to minimize that to keep the
footprint of possible class friction low. Native libs though...argh...dont
afford much flexibility with the way they're handled as far as we can tell just
yet.
> Nifi JMS Controller Disable/Enable causes Classloader Issues with Native Libs
> -----------------------------------------------------------------------------
>
> Key: NIFI-5184
> URL: https://issues.apache.org/jira/browse/NIFI-5184
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.5.0
> Reporter: Greg Senia
> Priority: Major
>
> When attempting to use a Local IBM QMGR with MQ Bindings. IBM States that
> within the JDK only one instance of the native library can be loaded. I've
> worked around this issue by symlinking the IBM MQ Libs into nifi/lib/ which
> is not a good solution. Wondering if this is a known issue with Nifi and the
> NAR classloader functions or if this is something that can be fixed so that
> Nifi can correctly work IBM MQ and MQBindings.
>
> This only occurs after you disable and than reenable the controller:
>
> 2018-05-10 21:10:34,865 ERROR [Timer-Driven Process Thread-2]
> o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS -
> JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS -
> JMSConsumer[destination:null; pub-sub:false;] failed to process session due
> to org.springframework.jms.UncategorizedJmsException: Uncategorized exception
> occured during JMS processing; nested exception is
> com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An exception
> occurred in the Java(tm) MQI.
> The Java(tm) MQI has thrown an exception describing the problem.
> See the linked exception for further information.; nested exception is
> com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to load the
> WebSphere MQ native JNI library: 'mqjbnd'.: {}
> org.springframework.jms.UncategorizedJmsException: Uncategorized exception
> occured during JMS processing; nested exception is
> com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An exception
> occurred in the Java(tm) MQI.
> The Java(tm) MQI has thrown an exception describing the problem.
> See the linked exception for further information.; nested exception is
> com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to load the
> WebSphere MQ native JNI library: 'mqjbnd'.
> at
> org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
> at
> org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:169)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:497)
> at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
> at
> org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
> at
> org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
> at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An
> exception occurred in the Java(tm) MQI.
> at sun.reflect.GeneratedConstructorAccessor192.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
> at
> com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8445)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:7814)
> at
> com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:299)
> at
> com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)
> at
> com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6024)
> at
> com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6049)
> at
> org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:181)
> at
> org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:152)
> at
> org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:365)
> at
> org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:305)
> at
> org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:283)
> at
> org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:224)
> at
> org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:484)
> ... 16 common frames omitted
> Caused by: com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to
> load the WebSphere MQ native JNI library: 'mqjbnd'.
> at com.ibm.mq.jmqi.local.LocalMQ.loadLib(LocalMQ.java:1268)
> at com.ibm.mq.jmqi.local.LocalMQ$1.run(LocalMQ.java:309)
> at java.security.AccessController.doPrivileged(Native Method)
> at com.ibm.mq.jmqi.local.LocalMQ.initialise_inner(LocalMQ.java:259)
> at com.ibm.mq.jmqi.local.LocalMQ.initialise(LocalMQ.java:221)
> at com.ibm.mq.jmqi.local.LocalMQ.<init>(LocalMQ.java:1350)
> at com.ibm.mq.jmqi.local.LocalServer.<init>(LocalServer.java:230)
> at sun.reflect.GeneratedConstructorAccessor191.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at com.ibm.mq.jmqi.JmqiEnvironment.getInstance(JmqiEnvironment.java:706)
> at com.ibm.mq.jmqi.JmqiEnvironment.getMQI(JmqiEnvironment.java:640)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8437)
> ... 29 common frames omitted
> Caused by: java.lang.UnsatisfiedLinkError: Native Library
> /opt/mqm/java/lib64/libmqjbnd.so already loaded in another classloader
> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1907)
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
> at java.lang.Runtime.load0(Runtime.java:809)
> at java.lang.System.load(System.java:1086)
> at com.ibm.mq.jmqi.local.LocalMQ.loadLib(LocalMQ.java:1228)
> ... 41 common frames omitted
>
>
>
> I've worked around this by doing the following:
> -rw-r-----. 1 nifi nifi 8073242 Jan 30 18:23 nifi-livy-nar-1.5.0.nar
> -rw-r-----. 1 nifi nifi 65753907 Jan 30 18:23 nifi-atlas-nar-1.5.0.nar
> drwxr-xr-x. 7 nifi nifi 140 May 9 21:43 ..
> drwxr-xr-x. 2 nifi nifi 4096 May 10 22:18 bootstrap
> lrwxrwxrwx. 1 nifi nifi 42 May 10 22:25 com.ibm.mq.allclient.jar ->
> /opt/mqm/java/lib/com.ibm.mq.allclient.jar
> lrwxrwxrwx. 1 nifi nifi 38 May 10 22:25 com.ibm.mq.axis2.jar ->
> /opt/mqm/java/lib/com.ibm.mq.axis2.jar
> lrwxrwxrwx. 1 nifi nifi 40 May 10 22:25 com.ibm.mq.headers.jar ->
> /opt/mqm/java/lib/com.ibm.mq.headers.jar
> lrwxrwxrwx. 1 nifi nifi 32 May 10 22:25 com.ibm.mq.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jar
> lrwxrwxrwx. 1 nifi nifi 37 May 10 22:25 com.ibm.mq.jmqi.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jmqi.jar
> lrwxrwxrwx. 1 nifi nifi 35 May 10 22:25 com.ibm.mqjms.jar ->
> /opt/mqm/java/lib/com.ibm.mqjms.jar
> lrwxrwxrwx. 1 nifi nifi 43 May 10 22:25 com.ibm.mq.jms.Nojndi.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jms.Nojndi.jar
> lrwxrwxrwx. 1 nifi nifi 36 May 10 22:25 com.ibm.mq.pcf.jar ->
> /opt/mqm/java/lib/com.ibm.mq.pcf.jar
> lrwxrwxrwx. 1 nifi nifi 37 May 10 22:25 com.ibm.mq.soap.jar ->
> /opt/mqm/java/lib/com.ibm.mq.soap.jar
> lrwxrwxrwx. 1 nifi nifi 45 May 10 22:25 com.ibm.mq.traceControl.jar ->
> /opt/mqm/java/lib/com.ibm.mq.traceControl.jar
> lrwxrwxrwx. 1 nifi nifi 31 May 10 22:25 fscontext.jar ->
> /opt/mqm/java/lib/fscontext.jar
> lrwxrwxrwx. 1 nifi nifi 25 May 10 22:25 jms.jar ->
> /opt/mqm/java/lib/jms.jar
> lrwxrwxrwx. 1 nifi nifi 34 May 10 22:25 providerutil.jar ->
> /opt/mqm/java/lib/providerutil.jar
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
