Thank you for the quick response. We upgraded the mina-filter-ssl-1.1.0.jar to mina-filter-ssl-1.1.1-20070611.145558-5.jar to see if it helped a day ago. Upgrading to mina-filter-ssl-1.1.1-20070611.145558-5.jar did address our SSL Handshake hang issue, but than this issue happened. Do you have clients that pipeline requests, which could expose a similar concurrency problem? I know that this problem does not occur with non-pipelining clients.
I'm taking your advice and upgrading to the official mina-core-1.1.1.jar and mina-filter-ssl-1.1.1.jar to see it that makes a difference. I'll post my findings after trying it out. Maarten Bosteels-4 wrote: > > James, > > People have reported deadlocks in SSLFilter before. > We are using the SSLFilter for one and a half year without issues, but > that's probably because > we're not writing to the IoSession from multiple threads simultaneously. > > Trustin has fixed at least two bugs in SSLFilter since 2007-06-11 > therefor I suggest you try with the latest release mina-1.1.1 which was > released on July 18th, 2007. > > https://issues.apache.org/jira/browse/DIRMINA-390 > https://issues.apache.org/jira/browse/DIRMINA-402 > > By the way, the version of jconsole that comes with jdk1.6 has a "Detect > Deadlock" button, have you tried that ? > > Maarten > > On 7/27/07, James Gould <[EMAIL PROTECTED]> wrote: >> >> >> I've deployed a Mina-based SSL Gateway that uses a remote client >> IoAcceptor >> with the SSLFilter, called the source channel, linked to an IoConnector >> that >> does not include the SSLFilter, called the destination channel, to TCP >> communicate to a back-end Application Server. The IoSession of the >> source >> and destination channels are held in a PipedSession where the source can >> push packets to the destination and vice-versa. We're coming across a >> synchronization issue, where the Gateway will run for over an hour and >> than >> will lock up. We incorporated the ExecutorFilter for both the source and >> destination to reduce the likelihood of the lock up, but it eventually >> will >> lock up the threads in the source and destination thread pools. >> >> We've taken a thread dump of the Gateway in the bad state and the >> following >> is a typical scenario, where the client might be pipelining. In >> pipelining, >> the client could send packets without waiting for the response of each >> packet. What we see is the source decrypting the request packet and the >> destination attempting to get a lock on the SSLHandler to encrypt the >> response back to the client. The issue is that the source is waiting on >> a >> lock in the SSLEngineImpl.unwrap of the Sun JDK 6.0 U2. The thread dump >> does not show who has the SSLEngineImpl.unwrap lock, but I believe that >> whoever has the lock can't release it due to a deadlock scenario. An >> example of the thread dump entries of the scenario are included. I'm >> reviewing the SSLFilter and SSLHandler, I'm not sure what code path could >> cause such a scenario. >> >> To note, we're running Mina 1.1.0 with >> mina-filter-ssl-1.1.1-20070611.145558-5.jar, since using the >> mina-filter-1.1.0.jar caused lockup issues with the SSL handshake. We >> deployed mina-filter-ssl-1.1.1-20070611.145558-5.jar to address the SSL >> handshake induced lockup. >> >> Is anyone else have similar SSL synchronization/deadlock issues? Any >> ideas >> or suggestions is greatly appreciated. >> >> The thread dump does not include what thread has the lock for < >> 0x94f12f98 >> >. >> >> "pool-2-thread-9" prio=10 tid=0x083f6c00 nid=0x2b7a runnable >> [0x8f445000..0x8f4461b0] >> java.lang.Thread.State: RUNNABLE >> at >> com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:667) >> - waiting to lock <0x94f12f98> (a java.lang.Object) >> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:607) >> at org.apache.mina.filter.support.SSLHandler.unwrap( >> SSLHandler.java:659) >> at org.apache.mina.filter.support.SSLHandler.decrypt( >> SSLHandler.java:465) >> at >> org.apache.mina.filter.support.SSLHandler.messageReceived(SSLHandler.java >> :334) >> at >> org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java >> :411) >> - locked <0x94f12c10> (a org.apache.mina.filter.support.SSLHandler) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived >> (AbstractIoFilterChain.java:362) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived >> (AbstractIoFilterChain.java:800) >> at >> org.apache.mina.common.IoFilterAdapter.messageReceived( >> IoFilterAdapter.java:83) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived >> (AbstractIoFilterChain.java:362) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived >> (AbstractIoFilterChain.java:800) >> at >> org.apache.mina.filter.executor.ExecutorFilter.processEvent( >> ExecutorFilter.java:247) >> at >> org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run( >> ExecutorFilter.java:307) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask( >> ThreadPoolExecutor.java:885) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >> :907) >> at java.lang.Thread.run(Thread.java:619) >> >> "pool-4-thread-33" prio=10 tid=0x083c3800 nid=0x2bef waiting for monitor >> entry [0x8e056000..0x8e0570b0] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at org.apache.mina.filter.SSLFilter.filterWrite(SSLFilter.java >> :478) >> - waiting to lock <0x94f12c10> (a >> org.apache.mina.filter.support.SSLHandler) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite >> (AbstractIoFilterChain.java:445) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite >> (AbstractIoFilterChain.java:824) >> at >> com.verisign.mina.filter.SetSessionIdInLogFilter.filterWrite( >> SetSessionIdInLogFilter.java:102) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite >> (AbstractIoFilterChain.java:445) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite >> (AbstractIoFilterChain.java:824) >> at >> org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite( >> ProtocolCodecFilter.java:227) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite >> (AbstractIoFilterChain.java:445) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite >> (AbstractIoFilterChain.java:824) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.filterWrite >> (AbstractIoFilterChain.java:727) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite >> (AbstractIoFilterChain.java:445) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.fireFilterWrite( >> AbstractIoFilterChain.java:436) >> at >> org.apache.mina.transport.socket.nio.SocketSessionImpl.write0( >> SocketSessionImpl.java:196) >> at >> org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:149) >> at >> org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:135) >> at >> com.verisign.cag.protocol.common.PipedSession.write(PipedSession.java:78) >> at >> com.verisign.cag.protocol.common.SourceSession.write(SourceSession.java >> :371) >> at >> com.verisign.cag.protocol.common.SourceSession.messageReceived( >> SourceSession.java:245) >> at >> com.verisign.cag.protocol.srsepp.SRSEPPDestinationHandler.messageReceived( >> SRSEPPDestinationHandler.java:68) >> at >> com.verisign.cag.protocol.srsepp.SRSEPPDestinationHandler.messageReceived( >> SRSEPPDestinationHandler.java:48) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived >> (AbstractIoFilterChain.java:703) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived >> (AbstractIoFilterChain.java:362) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived >> (AbstractIoFilterChain.java:800) >> at >> org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush( >> SimpleProtocolDecoderOutput.java:62) >> at >> org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived( >> ProtocolCodecFilter.java:200) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived >> (AbstractIoFilterChain.java:362) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived >> (AbstractIoFilterChain.java:800) >> at >> com.verisign.mina.filter.SetSessionIdInLogFilter.messageReceived( >> SetSessionIdInLogFilter.java:84) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived >> (AbstractIoFilterChain.java:362) >> at >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( >> AbstractIoFilterChain.java:54) >> at >> >> org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived >> (AbstractIoFilterChain.java:800) >> at >> org.apache.mina.filter.executor.ExecutorFilter.processEvent( >> ExecutorFilter.java:247) >> at >> org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run( >> ExecutorFilter.java:307) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask( >> ThreadPoolExecutor.java:885) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >> :907) >> at java.lang.Thread.run(Thread.java:619) >> >> >> -- >> View this message in context: >> http://www.nabble.com/Deadlock-with-Mina-based-SSL-Gateway-tf4159854s16868.html#a11835745 >> Sent from the Apache MINA Support Forum mailing list archive at >> Nabble.com >> . >> >> > > -- View this message in context: http://www.nabble.com/Deadlock-with-Mina-based-SSL-Gateway-tf4159854s16868.html#a11837353 Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.
