Please make a Jira. I also need an example to run which reproduces the
error.
On Tue, Apr 25, 2023 at 6:35 AM Kishore Mokkarala <kishore....@gmail.com>
wrote:
> Hi,
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at awaitUninterruptibly. Once
> this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.
>
> *Here is the sample thread dump:*
> "pool-118-thread-6" #508 prio=5 os_prio=0 cpu=345.84ms elapsed=*1929.48s*
> tid=0x00007ec6fc001800 nid=0x4b5d4 in Object.wait() [0x00007ec7792d4000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.14.1/Native Method)
> - waiting on <no object reference available>
> at
>
> org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:218)
> - waiting to re-lock in wait() <0x00007ed90d1b2c40> (a
> org.apache.mina.core.polling.AbstractPollingIoConnector$ConnectionRequest)
> at
>
> org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:148)
> at
>
> org.apache.mina.core.future.DefaultConnectFuture.awaitUninterruptibly(DefaultConnectFuture.java:149)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.service.G10CaptureService.startRecordCapture(G10CaptureService.java:622)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.geoblade.service.GeoBladeCaptureService.startRecordCapture(GeoBladeCaptureService.java:67)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.processGatewaySrQueryResponseSuccess(G10PluginCaptureProcessor.java:2156)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.doHandleGatewaySrQueryResponse(G10MinaClient.java:283)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.handleGatewaySrQueryResponse(G10MinaClient.java:268)
> at jdk.internal.reflect.GeneratedMethodAccessor201.invoke(Unknown
> Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(
> java.base@11.0.14.1/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(
> java.base@11.0.14.1/Method.java:566)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232)
> at
>
> org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:273)
> at
>
> org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158)
> - locked <0x00007ed92a951cd8> (a
> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
> at
>
> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
> at com.sun.proxy.$Proxy85.messageReceived(Unknown Source)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> org.apache.mina.filter.keepalive.KeepAliveFilter.messageReceived(KeepAliveFilter.java:414)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.messageReceived(G10GPBMessageIoFilter.java:100)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
> org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:106)
> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:89)
> at
>
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:763)
> at
>
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:755)
> at
>
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:695)
> at java.lang.Thread.run(java.base@11.0.14.1/Thread.java:829)
>
>
> "pool-116-thread-8" #458 prio=5 os_prio=0 cpu=172.16ms elapsed=529.47s
> tid=0x00007ed6c800e000 nid=0x124b in Object.wait() [0x00007ed6d9cda000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.14.1/Native Method)
> - waiting on <no object reference available>
> at
>
> org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:218)
> - waiting to re-lock in wait() <0x00007ee3a971c170> (a
> org.apache.mina.core.polling.AbstractPollingIoConnector$ConnectionRequest)
> at
>
> org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:148)
> at
>
> org.apache.mina.core.future.DefaultConnectFuture.awaitUninterruptibly(DefaultConnectFuture.java:149)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.service.G10CaptureService.startRecordCapture(G10CaptureService.java:622)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.geoblade.service.GeoBladeCaptureService.startRecordCapture(GeoBladeCaptureService.java:67)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.processGatewaySrQueryResponseSuccess(G10PluginCaptureProcessor.java:2156)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.doHandleGatewaySrQueryResponse(G10MinaClient.java:283)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.handleGatewaySrQueryResponse(G10MinaClient.java:268)
> at jdk.internal.reflect.GeneratedMethodAccessor281.invoke(Unknown
> Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(
> java.base@11.0.14.1/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(
> java.base@11.0.14.1/Method.java:566)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232)
> at
>
> org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:273)
> at
>
> org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158)
> - locked <0x00007ee3aaed5d58> (a
> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
> at
>
> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
> at com.sun.proxy.$Proxy85.messageReceived(Unknown Source)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> org.apache.mina.filter.keepalive.KeepAliveFilter.messageReceived(KeepAliveFilter.java:414)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.messageReceived(G10GPBMessageIoFilter.java:100)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
> org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:106)
> at org.apache.mina.core.session.IoEvent.run(IoEvent.java:89)
> at
>
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:763)
>
> *This code is working fine with 2.0.21.*
>
> *Here is the code snippet.*
>
> private static final ExecutorFilter executorFilter = *new
> ExecutorFilter(16,32);*
>
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
>
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> connectionLock.lock(); // connectionLock is private variable.
> try{
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> primaryConnectFuture.awaitUninterruptibly(); // no timeout specified.
>
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression,
> captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }
> finally
> {
> connectionLock.unLock();
> }
>
> ------------------------------------------
> M.V.S.Kishore
> 91-9886412814
>
