Hi Emmanuel, Same code is working fine with 2.0.21, There is no deadlock, we have not done any code changes. To make it compatible with 2.2.1 we did some https implementation changes to our custom SSL filter. Last time I raised my concern on removing PEER_ADDRESS(in 2.2.1) for passing SNI_HOST_NAME during SSL handshake. Mina core 2.0.21 has some security vulnerabilities so we tested with 2.0.23, that time your support team suggested us to migrate to latest mina version 2.2.1, With this we stuck with dead lock,I am sure after changing Https implementation we are facing this issue,Here is the code we used in CustomSSLFilter for passing SNI hostname.
* 2.0.21 implementation:* public class CustomSslFilter extends SslFilter { public CustomSslFilter(SSLContext sslContext) { super(sslContext); LOGGER.info("Apache mina 2.0.X https implementation..."); } public void onPreAdd(IoFilterChain parent, String name, NextFilter nextFilter) throws SSLException { // Check that we don't have a SSL filter already present in the chain if (parent.contains(SslFilter.class)) { String msg = "Only one SSL filter is permitted in a chain."; LOGGER.error(msg); throw new IllegalStateException(msg); } IoSession session = parent.getSession(); Provider provider = (Provider)session.getAttribute(G10MinaClient.PROVIDER_KEY); * //Passing SNI hostname during handshake * InetSocketAddress probeAddress = InetSocketAddress.createUnresolved(SNIHostNames.DELIVERY_POINT.getHostName(),Integer.parseInt(provider.getProbe().getPortNumber())); * session.setAttribute(PEER_ADDRESS, probeAddress);* super.onPreAdd(parent, name, nextFilter); } } * 2.2.1 implementation:* public class CustomSslFilter extends SslFilter { public CustomSslFilter(SSLContext sslContext) { super(sslContext); LOGGER.info("Apache mina 2.2.3 https implementation..."); } protected SSLEngine createEngine(IoSession session, InetSocketAddress addr) { Provider provider = (Provider)session.getAttribute(G10MinaClient.PROVIDER_KEY); *//Passing SNI hostname during handshake * InetSocketAddress peer = InetSocketAddress.createUnresolved(SNIHostNames.DELIVERY_POINT.getHostName(),Integer.parseInt(provider.getProbe().getPortNumber())); SSLEngine sslEngine = (addr != null) ? sslContext.createSSLEngine(peer.getHostString(), peer.getPort()) : sslContext.createSSLEngine(); if (wantClientAuth) { sslEngine.setWantClientAuth(true); } if (needClientAuth) { sslEngine.setNeedClientAuth(true); } if (enabledCipherSuites != null) { sslEngine.setEnabledCipherSuites(enabledCipherSuites); } if (enabledProtocols != null) { sslEngine.setEnabledProtocols(enabledProtocols); } sslEngine.setUseClientMode(!session.isServer()); if (session.getAttribute(G10MinaClient.SESSION_ID_KEY) !=null ) { LOGGER.info(session.getAttribute(G10MinaClient.SESSION_ID_KEY) + " Adding the SSL Filter to the chain for " + provider.getProbeDisplayName()); } else { LOGGER.info("Adding the SSL Filter to the chain for " + provider.getProbeDisplayName()); } return sslEngine; } } I will appreciate your help in advance. ------------------------------------------ M.V.S.Kishore 91-9886412814 On Fri, 29 Dec 2023 at 13:38, Kishore Mokkarala <kishore....@gmail.com> wrote: > Hi Emmanuel, > Any help would be greatly appreciated. > ------------------------------------------ > M.V.S.Kishore > 91-9886412814 > > > On Thu, 28 Dec 2023 at 13:34, Kishore Mokkarala <kishore....@gmail.com> > wrote: > >> >> Thanks Emmanuel,I appreciate your quick response. I have one quick >> question as you said "*finish_handshake method will prob-pagate the >> Session_secured event to the IoHandler"* how to incorporate this code >> in my state machine code ? >> Currently we are on mina 2.2.1 code. Also i am sharing some sudo code too. >> >> public class G10MinaClient { >> >> private final G10CaptureProcessor captureProcessor; >> >> public G10MinaClient(final G10CaptureProcessor captureProcessor) { >> this.captureProcessor = captureProcessor; >> } >> >> /** >> * This method is called when a new session is created. >> * >> * @param context Context for this state machine session. >> * @param session The current IO session. >> */ >> @IoHandlerTransition(on = SESSION_CREATED, in = CONNECTED) >> public void create(final G10StateContext context, final IoSession >> session) { >> captureProcessor.sessionCreated(context, session); >> } >> >> >> @IoHandlerTransition(on = SESSION_CLOSED, in = OPENED) >> public void sessionClose(final G10StateContext context, final >> IoSession session) { >> captureProcessor.sessionClosed(context, session, false); >> } >> >> @IoHandlerTransition(on = SESSION_OPENED, in = CONNECTED) >> public void connect(final G10StateContext context, final IoSession >> session) { >> //This will write data on to SSL socket >> //Here how to check hand shake is over or not ? >> captureProcessor.verifyAndSendStartMsgs(context, session); >> } >> } >> >> *State machine,IO handler and Filter Chain creation:* >> >> StateMachine stateMachine = >> StateMachineFactory.getInstance(IoHandlerTransition.class).create( >> G10MinaClient.CONNECTED, new G10MinaClient(processor)); >> >> IoHandler ioHandler = new >> StateMachineProxyBuilder().setStateContextLookup( >> new IoSessionStateContextLookup(new StateContextFactory() >> { >> @Override >> public StateContext create() { >> final G10StateContext stateContext = new >> G10StateContext(); >> stateContext.setStartedTime(new Date()); >> LOG.info("G10StateContext initialized at:{} >> ",System.currentTimeMillis()); >> return stateContext; >> } >> })).create(IoHandler.class, stateMachine); >> >> >> NioSocketConnector connector = new NioSocketConnector(); >> connector.getFilterChain().addLast("LoggingFilter", >> G10CaptureService.loggingFilter); >> connector.getFilterChain().addLast("codecFilter", >> G10CaptureService.probeCodecFilter); >> connector.getFilterChain().addLast("executorFilter", >> G10CaptureService.executorFilter); >> connector.getFilterChain().addLast("gpbMessageFilter", >> G10CaptureService.gpbMessageFilter); >> connector.getFilterChain().addLast("keepAliveFilter", >> G10CaptureService.keepAliveFilter); >> >> SslFilter sslFilter; >> try { >> SSLContext sslContext = TLSUtil.getSSLContext(); >> sslFilter = new CustomSslFilter(sslContext); >> connector.getFilterChain().addFirst("sslFilter", sslFilter); >> } catch (Exception e) { >> e.printStackTrace(); >> LOG.error("Exception during creating SSL context..." + >> XError.getStackTrace(e)); >> } >> connector.setHandler(ioHandler); >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------ >> M.V.S.Kishore >> >> >> On Thu, 28 Dec 2023 at 04:29, Emmanuel Lécharny <elecha...@gmail.com> >> wrote: >> >>> Hi, >>> >>> what I can see from the 2 blocked threads is that one his holding a lock >>> on the SM context, then tries to write something over a TLS connection, >>> while the other receives something through the TLS connection, and tries >>> to process a SM event. >>> >>> There is a clear inter-blockage, with thread 1 doing: >>> - hold lock on the SSLHandlerG0 instance in the finish_handshake >>> function (which is synchronized) >>> - get a lock on the SM context >>> >>> and thread 2 doing: >>> - hold a lock on the SM contect >>> - and tries to get a lock on the SSLHandlerG0 instance to write >>> something that is generated in the verifyAndSendStartMsgs function. >>> >>> The finish_handshake method will prob-pagate the Session_secured event >>> to the IoHandler, and I think it's a good idea to handle it before >>> trying to write anything. That means you need to avoid writing anything >>> until the session_secured event has been handled. >>> >>> I wonder if the verifyAndSendStartMsgs should not be sent in >>> session-secured event rather than in the session-opened... >>> >>> >>> On 27/12/2023 17:30, Kishore Mokkarala wrote: >>> > Hi Emmanuel, >>> > >>> > You can use the attached thread dump to analyze.hope it will load.This >>> > is generated using jstack. >>> > ------------------------------------------ >>> > M.V.S.Kishore >>> > Lead Dev Engineer >>> > NetScout S/w Pvt. Ltd. >>> > 91-9886412814 >>> > >>> > >>> > On Wed, 27 Dec 2023 at 14:47, Kishore Mokkarala <kishore....@gmail.com >>> > <mailto:kishore....@gmail.com>> wrote: >>> > >>> > Here is the thread dump. >>> > >>> > Regards, >>> > ------------------------------------------ >>> > M.V.S.Kishore >>> > 91-9886412814 >>> > >>> > >>> > On Wed, 27 Dec 2023 at 10:59, Kishore Mokkarala >>> > <kishore....@gmail.com <mailto:kishore....@gmail.com>> wrote: >>> > >>> > We cannot share the source code,i have tried with latest Mina >>> > version 2.2.3 also,this is also not working.Dead lock is >>> > happening.Our Organization won't allow to share the source >>> > code.I am sure there is an issue with the state machine and >>> > Apache mina SSL used together.Please let me know how to proceed >>> > on this further ? i am open to zoom or webex call . >>> > >>> > Thanks & Regards, >>> > M.V.S.Kishore >>> > 91-9886412814 >>> > >>> > >>> > On Sun, 24 Dec 2023 at 08:06, Emmanuel Lécharny >>> > <elecha...@gmail.com <mailto:elecha...@gmail.com>> wrote: >>> > >>> > Can you provide the code you use that is related to the >>> > StateMachine usage? >>> > >>> > On 24/12/2023 03:22, Emmanuel Lécharny wrote: >>> > > Hi, >>> > > >>> > > the problem is most certainly in the StateMachine part. >>> I >>> > have to see >>> > > what it does. >>> > > >>> > > On 22/12/2023 17:13, Kishore Mokkarala wrote: >>> > >> *Hi Emmanuel,* >>> > >> * >>> > >> * >>> > >> *Deadlock is happening in the 3rd party jars,not within >>> > the code,which >>> > >> is out of the scope of the business logic. When large >>> > data is being >>> > >> transferred between two entities with parallel TCP ip >>> > connections >>> > >> **over SSL connection.** This is happening * >>> > >> *Could you please provide a work around this problem,we >>> > are kind of >>> > >> stuck now in the production.* >>> > >> *Currently we are using mina version 2.2.1.* >>> > >> * >>> > >> * >>> > >> *StateMachine.java:138* >>> > >> >>> > >> *public* *void* handle(Event >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/event/Event.html#Event >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/event/Event.html#Event>> >>> event) { >>> > >> 136 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L136 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L136>> >>> StateContext < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/context/StateContext.html#StateContext >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/context/StateContext.html#StateContext>> >>> context = event.getContext(); >>> > >> 137 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L137 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L137>> >>> 138 < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L138 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L138>> >>> *synchronized (context) {* >>> > >> 139 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L139 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L139>> >>> LinkedList<Event> eventQueue = eventQueueThreadLocal.get(); >>> > >> 140 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L140 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L140>> >>> eventQueue.addLast(event); >>> > >> 141 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L141 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L141>> >>> 142 < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L142 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L142>> >>> *if* (processingThreadLocal.get()) { >>> > >> 143 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L143 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L143>> >>> //*/ >>> > >> 144 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L144 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L144>> >>> /* This thread is already processing an event. Queue this / >>> > >> 145 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L145 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L145>> >>> /* event./ >>> > >> 146 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L146 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L146>> >>> /*// >>> > >> 147 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L147 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L147>> >>> *if* (LOGGER.isDebugEnabled()) { >>> > >> 148 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L148 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L148>> >>> LOGGER.debug("State machine called recursively. Queuing event k{} for later >>> processing.", event); >>> > >> 149 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L149 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L149>> >>> } >>> > >> 150 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L150 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L150>> >>> }*else* { >>> > >> 151 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L151 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L151>> >>> processingThreadLocal.set(*true*); >>> > >> 152 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L152 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L152>> >>> 153 < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L153 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L153>> >>> *try* { >>> > >> 154 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L154 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L154>> >>> *if* (context.getCurrentState() ==*null*) { >>> > >> 155 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L155 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L155>> >>> context.setCurrentState(startState); >>> > >> 156 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L156 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L156>> >>> } >>> > >> 157 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L157 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L157>> >>> 158 < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L158 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L158>> >>> processEvents(eventQueue); >>> > >> 159 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L159 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L159>> >>> }*finally* { >>> > >> 160 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L160 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L160>> >>> processingThreadLocal.set(false); >>> > >> 161 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L161 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L161>> >>> } >>> > >> 162 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L162 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L162>> >>> } >>> > >> 163 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L163 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L163>> >>> } >>> > >> 164 >>> > >> >>> > < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L164 >>> < >>> https://nightlies.apache.org/mina/mina/2.0.24/xref/org/apache/mina/statemachine/StateMachine.html#L164>> >>> } >>> > >> >>> > >> * >>> > >> * >>> > >> * at >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.write(SSLHandlerG0.java:312)* >>> > >> * >>> > >> * >>> > >> *SSLHandlerG0.java:* >>> > >> synchronized public void write(NextFilter next, >>> > WriteRequest request) >>> > >> throws SSLException, WriteRejectedException { >>> > >> if (LOGGER.isDebugEnabled()) { >>> > >> LOGGER.debug("{} write() - source {}", >>> > toString(), request); >>> > >> }* >>> > >> * >>> > >> ........................ >>> > >> .......................... >>> > >> } >>> > >> >>> > >> >>> > >> >>> > >> ------------------------------------------ >>> > >> M.V.S.Kishore >>> > >> Lead Dev Engineer >>> > >> NetScout S/w Pvt. Ltd. >>> > >> 91-9886412814 >>> > >> >>> > >> >>> > >> On Fri, 22 Dec 2023 at 20:10, Kishore Mokkarala >>> > <kishore....@gmail.com <mailto:kishore....@gmail.com> >>> > >> <mailto:kishore....@gmail.com >>> > <mailto:kishore....@gmail.com>>> wrote: >>> > >> >>> > >> 1. >>> > >> Is the below issue fixed ? >>> > >> 1. DIRMINA-604 >>> > >> <https://issues.apache.org/jira/browse/DIRMINA-604 >>> > <https://issues.apache.org/jira/browse/DIRMINA-604>> >>> > >> Deadlock occurs when implementing two mina >>> > StateMachine >>> > >> >>> > >> ------------------------------------------ >>> > >> M.V.S.Kishore >>> > >> Lead Dev Engineer >>> > >> NetScout S/w Pvt. Ltd. >>> > >> 91-9886412814 >>> > >> >>> > >> >>> > >> On Fri, 22 Dec 2023 at 13:04, Emmanuel Lécharny >>> > <elecha...@gmail.com <mailto:elecha...@gmail.com> >>> > >> <mailto:elecha...@gmail.com >>> > <mailto:elecha...@gmail.com>>> wrote: >>> > >> >>> > >> Hi, >>> > >> >>> > >> you have a deadlock on 0x00007f5c9f001408, >>> which >>> > is in >>> > >> >>> > >>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext. >>> > >> >>> > >> >>> > >> >>> > >> On 22/12/2023 08:10, Kishore Mokkarala wrote: >>> > >> > Hi Emmanuel, >>> > >> > >>> > >> > We are facing deadlock in production with >>> > mina 2.2.1 >>> > >> version,earlier it >>> > >> > was on 2.0.21, After upgrade only we are >>> > seeing this >>> > >> issue,kindly please >>> > >> > provide an update any similar kind issue is >>> > fixed in 2.2.3 or >>> > >> not ? >>> > >> > >>> > >> > >>> > >> > Thread pool-123-thread-3 is in deadlock with >>> > thread >>> > >> NioProcessor-37 >>> > >> > >>> > >> > >>> > >> > pool-123-thread-3 >>> > >> > >>> > >> > PRIORITY : 5 >>> > >> > >>> > >> > THREAD ID : 0X00007F5978002870 >>> > >> > >>> > >> > NATIVE ID : 0X7C15 >>> > >> > >>> > >> > NATIVE ID (DECIMAL) : 31765 >>> > >> > >>> > >> > STATE : BLOCKED >>> > >> > >>> > >> > >>> > >> > stackTrace: >>> > >> > java.lang.Thread.State: BLOCKED (on object >>> > monitor) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.write(SSLHandlerG0.java:312) >>> > >> > - waiting to lock <0x00007f5c9f001168> (a >>> > >> > org.apache.mina.filter.ssl.SSLHandlerG0) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SslFilter.filterWrite(SslFilter.java:380) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.filterWrite(IoFilterAdapter.java:138) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite(ProtocolCodecFilter.java:332) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.executor.ExecutorFilter.filterWrite(ExecutorFilter.java:595) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.filterWrite(G10GPBMessageIoFilter.java:63) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.filterWrite(IoFilterAdapter.java:138) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.filterWrite(IoFilterAdapter.java:138) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.fireFilterWrite(DefaultIoFilterChain.java:746) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.session.AbstractIoSession.write(AbstractIoSession.java:575) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.session.AbstractIoSession.write(AbstractIoSession.java:520) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.verifyAndSendStartMsgs(G10PluginCaptureProcessor.java:2627) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.sessionConnected(G10PluginCaptureProcessor.java:2552) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.connect(G10MinaClient.java:220) >>> > >> > at >>> > >> >>> > >>> jdk.internal.reflect.GeneratedMethodAccessor144.invoke(Unknown >>> > >> Source) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.7 >>> /DelegatingMethodAccessorImpl.java:43) >>> > >> > at >>> > >> >>> > java.lang.reflect.Method.invoke(java.base@17.0.7 >>> /Method.java:568) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158) >>> > >> > - locked <0x00007f5c9f001408> (a >>> > >> > >>> > >>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261) >>> > >> > at >>> > jdk.proxy4.$Proxy87.sessionOpened(jdk.proxy4/Unknown >>> > >> Source) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.sessionOpened(DefaultIoFilterChain.java:940) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionOpened(DefaultIoFilterChain.java:574) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$800(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionOpened(DefaultIoFilterChain.java:1083) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.sessionOpened(IoFilterAdapter.java:90) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionOpened(DefaultIoFilterChain.java:574) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$800(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionOpened(DefaultIoFilterChain.java:1083) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.sessionOpened(IoFilterAdapter.java:90) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionOpened(DefaultIoFilterChain.java:574) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$800(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionOpened(DefaultIoFilterChain.java:1083) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:127) >>> > >> > at >>> > org.apache.mina.core.session.IoEvent.run(IoEvent.java:89) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:763) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:755) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:695) >>> > >> > at >>> > java.lang.Thread.run(java.base@17.0.7/Thread.java:833) >>> > >> > >>> > >> > >>> > >> > NioProcessor-37 >>> > >> > >>> > >> > PRIORITY : 5 >>> > >> > >>> > >> > THREAD ID : 0X00007F5A040B5880 >>> > >> > >>> > >> > NATIVE ID : 0X6D19 >>> > >> > >>> > >> > NATIVE ID (DECIMAL) : 27929 >>> > >> > >>> > >> > STATE : BLOCKED >>> > >> > >>> > >> > >>> > >> > stackTrace: >>> > >> > java.lang.Thread.State: BLOCKED (on object >>> > monitor) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:138) >>> > >> > - waiting to lock <0x00007f5c9f001408> (a >>> > >> > >>> > >>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261) >>> > >> > at >>> > jdk.proxy4.$Proxy87.event(jdk.proxy4/Unknown Source) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.event(DefaultIoFilterChain.java:1039) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.filter.ssl.SSLHandlerG0.finish_handshake(SSLHandlerG0.java:589) >>> > >> > - locked <0x00007f5c9f001168> (a >>> > >> org.apache.mina.filter.ssl.SSLHandlerG0) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.receive_loop(SSLHandlerG0.java:271) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.receive_loop(SSLHandlerG0.java:246) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.receive_loop(SSLHandlerG0.java:246) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SSLHandlerG0.receive(SSLHandlerG0.java:162) >>> > >> > - locked <0x00007f5c9f001168> (a >>> > >> org.apache.mina.filter.ssl.SSLHandlerG0) >>> > >> > at >>> > >> >>> > >> >>> > >>> org.apache.mina.filter.ssl.SslFilter.messageReceived(SslFilter.java:342) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:122) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:643) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:539) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.polling.AbstractPollingIoProcessor.access$1200(AbstractPollingIoProcessor.java:68) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.process(AbstractPollingIoProcessor.java:1224) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.process(AbstractPollingIoProcessor.java:1213) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:683) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> >>> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.7 >>> /ThreadPoolExecutor.java:1136) >>> > >> > at >>> > >> > >>> > >> >>> > >> >>> > >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.7 >>> /ThreadPoolExecutor.java:635) >>> > >> > at >>> > java.lang.Thread.run(java.base@17.0.7/Thread.java:833) >>> > >> > >>> > >> > Regards, >>> > >> > ------------------------------------------ >>> > >> > M.V.S.Kishore >>> > >> > Lead Dev Engineer >>> > >> > NetScout S/w Pvt. Ltd. >>> > >> > 91-9886412814 >>> > >> >>> > >> -- *Emmanuel Lécharny* P. +33 (0)6 08 >>> 33 >>> > 32 61 >>> > >> elecha...@apache.org <mailto:elecha...@apache.org> >>> > <mailto:elecha...@apache.org <mailto:elecha...@apache.org >>> >> >>> > >> >>> > > >>> > >>> > -- >>> > *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61 >>> > elecha...@apache.org <mailto:elecha...@apache.org> >>> > >>> >>> -- >>> *Emmanuel Lécharny* P. +33 (0)6 08 33 32 61 >>> elecha...@apache.org >>> >>