[
https://issues.apache.org/jira/browse/NIFI-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471438#comment-16471438
]
Greg Senia commented on NIFI-5184:
----------------------------------
Is it possible to do something similar as the Spark guys did with
sharedPrefixes basically classes that should be shared within the JVM vs having
seperate classloaders for each instance?
| Creates a [[HiveClient]] using a classloader that works according to the
following rules:|
| | * - Shared classes: Java, Scala, logging, and Spark classes are delegated
to `baseClassLoader`|
| | * allowing the results of calls to the [[HiveClient]] to be visible
externally.|
| | * - Hive classes: new instances are loaded from `execJars`. These classes
are not|
| | * accessible externally due to their custom loading.|
| | * - [[HiveClientImpl]]: a new copy is created for each instance of
`IsolatedClassLoader`.|
| | * This new instance is able to see a specific version of hive without using
reflection. Since|
| | * this is a unique instance, it is not visible externally other than as a
generic|
| | * [[HiveClient]], unless `isolationOn` is set to `false`.|
> Nifi JMS Controller Disable/Enable causes Classloader Issues with Native Libs
> -----------------------------------------------------------------------------
>
> Key: NIFI-5184
> URL: https://issues.apache.org/jira/browse/NIFI-5184
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.5.0
> Reporter: Greg Senia
> Priority: Major
>
> When attempting to use a Local IBM QMGR with MQ Bindings. IBM States that
> within the JDK only one instance of the native library can be loaded. I've
> worked around this issue by symlinking the IBM MQ Libs into nifi/lib/ which
> is not a good solution. Wondering if this is a known issue with Nifi and the
> NAR classloader functions or if this is something that can be fixed so that
> Nifi can correctly work IBM MQ and MQBindings.
>
> This only occurs after you disable and than reenable the controller:
>
> 2018-05-10 21:10:34,865 ERROR [Timer-Driven Process Thread-2]
> o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS -
> JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS -
> JMSConsumer[destination:null; pub-sub:false;] failed to process session due
> to org.springframework.jms.UncategorizedJmsException: Uncategorized exception
> occured during JMS processing; nested exception is
> com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An exception
> occurred in the Java(tm) MQI.
> The Java(tm) MQI has thrown an exception describing the problem.
> See the linked exception for further information.; nested exception is
> com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to load the
> WebSphere MQ native JNI library: 'mqjbnd'.: {}
> org.springframework.jms.UncategorizedJmsException: Uncategorized exception
> occured during JMS processing; nested exception is
> com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An exception
> occurred in the Java(tm) MQI.
> The Java(tm) MQI has thrown an exception describing the problem.
> See the linked exception for further information.; nested exception is
> com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to load the
> WebSphere MQ native JNI library: 'mqjbnd'.
> at
> org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
> at
> org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:169)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:497)
> at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
> at
> org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
> at
> org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
> at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSFMQ6312: An
> exception occurred in the Java(tm) MQI.
> at sun.reflect.GeneratedConstructorAccessor192.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
> at
> com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8445)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:7814)
> at
> com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:299)
> at
> com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:236)
> at
> com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6024)
> at
> com.ibm.mq.jms.MQConnectionFactory.createConnection(MQConnectionFactory.java:6049)
> at
> org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:181)
> at
> org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:152)
> at
> org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:365)
> at
> org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:305)
> at
> org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:283)
> at
> org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:224)
> at
> org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:484)
> ... 16 common frames omitted
> Caused by: com.ibm.mq.jmqi.local.LocalMQ$4: CC=2;RC=2495;AMQ8598: Failed to
> load the WebSphere MQ native JNI library: 'mqjbnd'.
> at com.ibm.mq.jmqi.local.LocalMQ.loadLib(LocalMQ.java:1268)
> at com.ibm.mq.jmqi.local.LocalMQ$1.run(LocalMQ.java:309)
> at java.security.AccessController.doPrivileged(Native Method)
> at com.ibm.mq.jmqi.local.LocalMQ.initialise_inner(LocalMQ.java:259)
> at com.ibm.mq.jmqi.local.LocalMQ.initialise(LocalMQ.java:221)
> at com.ibm.mq.jmqi.local.LocalMQ.<init>(LocalMQ.java:1350)
> at com.ibm.mq.jmqi.local.LocalServer.<init>(LocalServer.java:230)
> at sun.reflect.GeneratedConstructorAccessor191.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at com.ibm.mq.jmqi.JmqiEnvironment.getInstance(JmqiEnvironment.java:706)
> at com.ibm.mq.jmqi.JmqiEnvironment.getMQI(JmqiEnvironment.java:640)
> at
> com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8437)
> ... 29 common frames omitted
> Caused by: java.lang.UnsatisfiedLinkError: Native Library
> /opt/mqm/java/lib64/libmqjbnd.so already loaded in another classloader
> at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1907)
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
> at java.lang.Runtime.load0(Runtime.java:809)
> at java.lang.System.load(System.java:1086)
> at com.ibm.mq.jmqi.local.LocalMQ.loadLib(LocalMQ.java:1228)
> ... 41 common frames omitted
>
>
>
> I've worked around this by doing the following:
> -rw-r-----. 1 nifi nifi 8073242 Jan 30 18:23 nifi-livy-nar-1.5.0.nar
> -rw-r-----. 1 nifi nifi 65753907 Jan 30 18:23 nifi-atlas-nar-1.5.0.nar
> drwxr-xr-x. 7 nifi nifi 140 May 9 21:43 ..
> drwxr-xr-x. 2 nifi nifi 4096 May 10 22:18 bootstrap
> lrwxrwxrwx. 1 nifi nifi 42 May 10 22:25 com.ibm.mq.allclient.jar ->
> /opt/mqm/java/lib/com.ibm.mq.allclient.jar
> lrwxrwxrwx. 1 nifi nifi 38 May 10 22:25 com.ibm.mq.axis2.jar ->
> /opt/mqm/java/lib/com.ibm.mq.axis2.jar
> lrwxrwxrwx. 1 nifi nifi 40 May 10 22:25 com.ibm.mq.headers.jar ->
> /opt/mqm/java/lib/com.ibm.mq.headers.jar
> lrwxrwxrwx. 1 nifi nifi 32 May 10 22:25 com.ibm.mq.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jar
> lrwxrwxrwx. 1 nifi nifi 37 May 10 22:25 com.ibm.mq.jmqi.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jmqi.jar
> lrwxrwxrwx. 1 nifi nifi 35 May 10 22:25 com.ibm.mqjms.jar ->
> /opt/mqm/java/lib/com.ibm.mqjms.jar
> lrwxrwxrwx. 1 nifi nifi 43 May 10 22:25 com.ibm.mq.jms.Nojndi.jar ->
> /opt/mqm/java/lib/com.ibm.mq.jms.Nojndi.jar
> lrwxrwxrwx. 1 nifi nifi 36 May 10 22:25 com.ibm.mq.pcf.jar ->
> /opt/mqm/java/lib/com.ibm.mq.pcf.jar
> lrwxrwxrwx. 1 nifi nifi 37 May 10 22:25 com.ibm.mq.soap.jar ->
> /opt/mqm/java/lib/com.ibm.mq.soap.jar
> lrwxrwxrwx. 1 nifi nifi 45 May 10 22:25 com.ibm.mq.traceControl.jar ->
> /opt/mqm/java/lib/com.ibm.mq.traceControl.jar
> lrwxrwxrwx. 1 nifi nifi 31 May 10 22:25 fscontext.jar ->
> /opt/mqm/java/lib/fscontext.jar
> lrwxrwxrwx. 1 nifi nifi 25 May 10 22:25 jms.jar ->
> /opt/mqm/java/lib/jms.jar
> lrwxrwxrwx. 1 nifi nifi 34 May 10 22:25 providerutil.jar ->
> /opt/mqm/java/lib/providerutil.jar
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
