On 7/28/07, James Gould <[EMAIL PROTECTED]> wrote: > > > Thank you for the quick response. We upgraded the > mina-filter-ssl-1.1.0.jar > to mina-filter-ssl-1.1.1-20070611.145558-5.jar to see if it helped a day > ago. Upgrading to mina-filter-ssl-1.1.1-20070611.145558-5.jar did address > our SSL Handshake hang issue, but than this issue happened. Do you have > clients that pipeline requests, which could expose a similar concurrency > problem? I know that this problem does not occur with non-pipelining > clients.
James, What exactly do you mean by "clients that pipeline requests" ? Do you mean clients that send multiple requests at once without waiting for their respective responses ? If that's what you mean, then yes, we most probably have clients doing this. But we don't process these requests concurrently. We are running a mina-based EPP server for some time now. I guess you are familiar with EPP :-) Maarten I'm taking your advice and upgrading to the official mina-core-1.1.1.jar and > mina-filter-ssl-1.1.1.jar to see it that makes a difference. I'll post my > findings after trying it out. > > > Maarten Bosteels-4 wrote: > > > > James, > > > > People have reported deadlocks in SSLFilter before. > > We are using the SSLFilter for one and a half year without issues, but > > that's probably because > > we're not writing to the IoSession from multiple threads simultaneously. > > > > Trustin has fixed at least two bugs in SSLFilter since 2007-06-11 > > therefor I suggest you try with the latest release mina-1.1.1 which was > > released on July 18th, 2007. > > > > https://issues.apache.org/jira/browse/DIRMINA-390 > > https://issues.apache.org/jira/browse/DIRMINA-402 > > > > By the way, the version of jconsole that comes with jdk1.6 has a "Detect > > Deadlock" button, have you tried that ? > > > > Maarten > > > > On 7/27/07, James Gould <[EMAIL PROTECTED]> wrote: > >> > >> > >> I've deployed a Mina-based SSL Gateway that uses a remote client > >> IoAcceptor > >> with the SSLFilter, called the source channel, linked to an IoConnector > >> that > >> does not include the SSLFilter, called the destination channel, to TCP > >> communicate to a back-end Application Server. The IoSession of the > >> source > >> and destination channels are held in a PipedSession where the source > can > >> push packets to the destination and vice-versa. We're coming across a > >> synchronization issue, where the Gateway will run for over an hour and > >> than > >> will lock up. We incorporated the ExecutorFilter for both the source > and > >> destination to reduce the likelihood of the lock up, but it eventually > >> will > >> lock up the threads in the source and destination thread pools. > >> > >> We've taken a thread dump of the Gateway in the bad state and the > >> following > >> is a typical scenario, where the client might be pipelining. In > >> pipelining, > >> the client could send packets without waiting for the response of each > >> packet. What we see is the source decrypting the request packet and > the > >> destination attempting to get a lock on the SSLHandler to encrypt the > >> response back to the client. The issue is that the source is waiting > on > >> a > >> lock in the SSLEngineImpl.unwrap of the Sun JDK 6.0 U2. The thread > dump > >> does not show who has the SSLEngineImpl.unwrap lock, but I believe that > >> whoever has the lock can't release it due to a deadlock scenario. An > >> example of the thread dump entries of the scenario are included. I'm > >> reviewing the SSLFilter and SSLHandler, I'm not sure what code path > could > >> cause such a scenario. > >> > >> To note, we're running Mina 1.1.0 with > >> mina-filter-ssl-1.1.1-20070611.145558-5.jar, since using the > >> mina-filter-1.1.0.jar caused lockup issues with the SSL handshake. We > >> deployed mina-filter-ssl-1.1.1-20070611.145558-5.jar to address the SSL > >> handshake induced lockup. > >> > >> Is anyone else have similar SSL synchronization/deadlock issues? Any > >> ideas > >> or suggestions is greatly appreciated. > >> > >> The thread dump does not include what thread has the lock for < > >> 0x94f12f98 > >> >. > >> > >> "pool-2-thread-9" prio=10 tid=0x083f6c00 nid=0x2b7a runnable > >> [0x8f445000..0x8f4461b0] > >> java.lang.Thread.State: RUNNABLE > >> at > >> com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java > :667) > >> - waiting to lock <0x94f12f98> (a java.lang.Object) > >> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:607) > >> at org.apache.mina.filter.support.SSLHandler.unwrap( > >> SSLHandler.java:659) > >> at org.apache.mina.filter.support.SSLHandler.decrypt( > >> SSLHandler.java:465) > >> at > >> org.apache.mina.filter.support.SSLHandler.messageReceived( > SSLHandler.java > >> :334) > >> at > >> org.apache.mina.filter.SSLFilter.messageReceived(SSLFilter.java > >> :411) > >> - locked <0x94f12c10> (a org.apache.mina.filter.support.SSLHandler) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived > >> (AbstractIoFilterChain.java:362) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived > >> (AbstractIoFilterChain.java:800) > >> at > >> org.apache.mina.common.IoFilterAdapter.messageReceived( > >> IoFilterAdapter.java:83) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived > >> (AbstractIoFilterChain.java:362) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived > >> (AbstractIoFilterChain.java:800) > >> at > >> org.apache.mina.filter.executor.ExecutorFilter.processEvent( > >> ExecutorFilter.java:247) > >> at > >> > org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run( > >> ExecutorFilter.java:307) > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask( > >> ThreadPoolExecutor.java:885) > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java > >> :907) > >> at java.lang.Thread.run(Thread.java:619) > >> > >> "pool-4-thread-33" prio=10 tid=0x083c3800 nid=0x2bef waiting for > monitor > >> entry [0x8e056000..0x8e0570b0] > >> java.lang.Thread.State: BLOCKED (on object monitor) > >> at org.apache.mina.filter.SSLFilter.filterWrite(SSLFilter.java > >> :478) > >> - waiting to lock <0x94f12c10> (a > >> org.apache.mina.filter.support.SSLHandler) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite > >> (AbstractIoFilterChain.java:445) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite > >> (AbstractIoFilterChain.java:824) > >> at > >> com.verisign.mina.filter.SetSessionIdInLogFilter.filterWrite( > >> SetSessionIdInLogFilter.java:102) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite > >> (AbstractIoFilterChain.java:445) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite > >> (AbstractIoFilterChain.java:824) > >> at > >> org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite( > >> ProtocolCodecFilter.java:227) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite > >> (AbstractIoFilterChain.java:445) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1400( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite > >> (AbstractIoFilterChain.java:824) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.filterWrite > >> (AbstractIoFilterChain.java:727) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite > >> (AbstractIoFilterChain.java:445) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.fireFilterWrite( > >> AbstractIoFilterChain.java:436) > >> at > >> org.apache.mina.transport.socket.nio.SocketSessionImpl.write0( > >> SocketSessionImpl.java:196) > >> at > >> org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java > :149) > >> at > >> org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java > :135) > >> at > >> com.verisign.cag.protocol.common.PipedSession.write(PipedSession.java > :78) > >> at > >> com.verisign.cag.protocol.common.SourceSession.write(SourceSession.java > >> :371) > >> at > >> com.verisign.cag.protocol.common.SourceSession.messageReceived( > >> SourceSession.java:245) > >> at > >> > com.verisign.cag.protocol.srsepp.SRSEPPDestinationHandler.messageReceived( > >> SRSEPPDestinationHandler.java:68) > >> at > >> > com.verisign.cag.protocol.srsepp.SRSEPPDestinationHandler.messageReceived( > >> SRSEPPDestinationHandler.java:48) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived > >> (AbstractIoFilterChain.java:703) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived > >> (AbstractIoFilterChain.java:362) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived > >> (AbstractIoFilterChain.java:800) > >> at > >> org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush( > >> SimpleProtocolDecoderOutput.java:62) > >> at > >> org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived( > >> ProtocolCodecFilter.java:200) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived > >> (AbstractIoFilterChain.java:362) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived > >> (AbstractIoFilterChain.java:800) > >> at > >> com.verisign.mina.filter.SetSessionIdInLogFilter.messageReceived( > >> SetSessionIdInLogFilter.java:84) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived > >> (AbstractIoFilterChain.java:362) > >> at > >> org.apache.mina.common.support.AbstractIoFilterChain.access$1100( > >> AbstractIoFilterChain.java:54) > >> at > >> > >> > org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived > >> (AbstractIoFilterChain.java:800) > >> at > >> org.apache.mina.filter.executor.ExecutorFilter.processEvent( > >> ExecutorFilter.java:247) > >> at > >> > org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run( > >> ExecutorFilter.java:307) > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask( > >> ThreadPoolExecutor.java:885) > >> at > >> java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java > >> :907) > >> at java.lang.Thread.run(Thread.java:619) > >> > >> > >> -- > >> View this message in context: > >> > http://www.nabble.com/Deadlock-with-Mina-based-SSL-Gateway-tf4159854s16868.html#a11835745 > >> Sent from the Apache MINA Support Forum mailing list archive at > >> Nabble.com > >> . > >> > >> > > > > > > -- > View this message in context: > http://www.nabble.com/Deadlock-with-Mina-based-SSL-Gateway-tf4159854s16868.html#a11837353 > Sent from the Apache MINA Support Forum mailing list archive at Nabble.com > . > >
