I all. I'm trying to continue my troobleshooting.. I downgrade to activemq 5.5.1 because we used this version production and it seems the problem is still observable. With the old logger, I have a trace now: with a concurrentModificationException.
Please, any help ? mai 14, 2019 3:32:26 PM org.apache.activemq.network.DemandForwardingBridgeSupport serviceRemoteException AVERTISSEMENT: Network connection between vm://broker1#0 and tcp:///127.0.0.1:50002 shutdown due to a remote error: java.util.ConcurrentModificationException mai 14, 2019 3:32:26 PM org.apache.activemq.broker.TransportConnector stop INFOS: Connector vm://broker1 Stopped mai 14, 2019 3:32:26 PM org.apache.activemq.network.DemandForwardingBridgeSupport stop INFOS: broker1 bridge to broker2 stopped mai 14, 2019 3:32:26 PM org.apache.activemq.network.DiscoveryNetworkConnector onServiceAdd INFOS: Establishing network connection from vm://broker1?async=false&network=true to tcp://127.0.0.1:50002?connectionTimeout=2000&soTimeout=2000 mai 14, 2019 3:32:26 PM org.apache.activemq.broker.TransportConnector start INFOS: Connector vm://broker1 Started mai 14, 2019 3:32:26 PM org.apache.activemq.network.DemandForwardingBridgeSupport startLocalBridge INFOS: Network connection between vm://broker1#4 and tcp:///127.0.0.1:50002(broker2) has been established. com.thalesaleniaspace.openscc.exporter.spring.jms.MockServiceException: A runtime exception occurred during the method invocation. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:104) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy9.call(Unknown Source) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.Caller.run(Caller.java:40) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.ActiveMQ3NodesTest.testOnQueue(ActiveMQ3NodesTest.java:58) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: org.springframework.remoting.RemoteTimeoutException: Receive timeout when calling service 'IMockService' method 'call' after 2000 ms at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.onReceiveTimeout(JmsInvokerProxyFactoryBean.java:140) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.executeRequest(JmsInvokerProxyFactoryBean.java:128) at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:198) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:90) ... 29 more com.thalesaleniaspace.openscc.exporter.spring.jms.MockServiceException: A runtime exception occurred during the method invocation. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:104) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy9.call(Unknown Source) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.Caller.run(Caller.java:40) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.ActiveMQ3NodesTest.testOnQueue(ActiveMQ3NodesTest.java:58) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: org.springframework.remoting.RemoteTimeoutException: Receive timeout when calling service 'IMockService' method 'call' after 2000 ms at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.onReceiveTimeout(JmsInvokerProxyFactoryBean.java:140) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.executeRequest(JmsInvokerProxyFactoryBean.java:128) at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:198) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:90) ... 29 more mai 14, 2019 3:32:31 PM org.apache.activemq.network.DemandForwardingBridgeSupport serviceRemoteException AVERTISSEMENT: Network connection between vm://broker1#4 and tcp:///127.0.0.1:50002 shutdown due to a remote error: java.util.ConcurrentModificationException com.thalesaleniaspace.openscc.exporter.spring.jms.MockServiceException: A runtime exception occurred during the method invocation. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:104) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy9.call(Unknown Source) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.Caller.run(Caller.java:40) at com.thalesaleniaspace.openscc.exporter.spring.jms.integration.ActiveMQ3NodesTest.testOnQueue(ActiveMQ3NodesTest.java:58) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) *Caused by: java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) at java.util.HashMap$EntryIterator.next(HashMap.java:1471) at java.util.HashMap$EntryIterator.next(HashMap.java:1469) at java.util.HashMap.putMapEntries(HashMap.java:511) at java.util.HashMap.<init>(HashMap.java:489) at org.slf4j.helpers.BasicMDCAdapter.getCopyOfContextMap(BasicMDCAdapter.java:130) at org.slf4j.MDC.getCopyOfContextMap(MDC.java:176) at org.apache.activemq.util.MDCHelper.getCopyOfContextMap(MDCHelper.java:30) at org.apache.activemq.thread.PooledTaskRunner.<init>(PooledTaskRunner.java:42) at org.apache.activemq.thread.TaskRunnerFactory.createTaskRunner(TaskRunnerFactory.java:80) at org.apache.activemq.ActiveMQSessionExecutor.wakeup(ActiveMQSessionExecutor.java:101) at org.apache.activemq.ActiveMQMessageConsumer.start(ActiveMQMessageConsumer.java:1339) at org.apache.activemq.ActiveMQMessageConsumer.<init>(ActiveMQMessageConsumer.java:261) at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1116) at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060) at org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973) at *com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.doExecuteRequest(JmsInvokerProxyFactoryBean.java:167) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.executeRequest(JmsInvokerProxyFactoryBean.java:124) at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:198) at com.thalesaleniaspace.openscc.exporter.spring.jms.redundancy.JmsInvokerProxyFactoryBean.invoke(JmsInvokerProxyFactoryBean.java:90) ... 29 more -- Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html