Hi,
I am a developer and maintainer of the QuickFIX/J project (https://github.com/quickfix-j/quickfixj)
and I have a question regarding NioSocketConnectors.
We are facing a problem when there is a process that constantly (every 30 seconds) tries to connect
to a counterparty and the connection is established but dropped shortly after. Then sometimes the
NioProcessors/NioSocketConnectors are not cleaned up properly. In the stack trace we see them
hanging in a call to dispose:
"NioProcessor-1140" #239 prio=5 os_prio=0 tid=0x0000000001fe1800 nid=0x2523 runnable
[0x00007f9c67e8f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000000f6699e60> (a sun.nio.ch.Util$3)
- locked <0x00000000f6699e50> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000000f6699c18> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at
org.apache.mina.transport.socket.nio.NioProcessor.select(NioProcessor.java:98)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1075)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
"NioSocketConnector-68" #238 prio=5 os_prio=0 tid=0x00007f9c70caf000 nid=0x2522 in Object.wait()
[0x00007f9c6af9f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at
org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:209)
- locked <0x00000000f66ac718> (a
org.apache.mina.core.future.DefaultIoFuture)
at
org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:141)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor.dispose(AbstractPollingIoProcessor.java:188)
at
org.apache.mina.core.service.SimpleIoProcessorPool.dispose(SimpleIoProcessorPool.java:329)
- locked <0x00000000f66ac750> (a java.lang.Object)
at
org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:582)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
It does not happen very often: about 5% of the connection attempts leave a
NioSocketConnector hanging.
It only seems to happen though when the connection is disconnected by
"javax.net.ssl.SSLHandshakeException: SSL handshake failed". Although there are cases when there is
no leak even on an SSLHandshakeException.
If the connection was reset "normally" by "java.io.IOException: Connection reset by peer" then the
leak does not seem to occur. It also does not occur when the connection is refused right away.
Since this seems to be related to SSL connections: is there something that we need to take care of
when using the SSL filter?
The code for the IoSessionInitiator can be found here:
https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/main/java/quickfix/mina/initiator/IoSessionInitiator.java
I have added some comments in this gist (starting with "chrjohn"):
https://gist.github.com/chrjohn/2671f06d80e8d917d9061b573477ec5b
I cannot rule out that we might be doing something wrong here, so any pointer
is appreciated. :)
Thanks in advance for your help and best regards,
Chris.
--
Christoph John
Development & Support
Direct: +49 241 557080-28
Mailto:[email protected]
http://www.macd.com <http://www.macd.com/>
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
MACD GmbH
Oppenhoffallee 103
D-52066 Aachen
Tel: +49 241 557080-0 | Fax: +49 241 557080-10
Amtsgericht Aachen: HRB 8151
Ust.-Id: DE 813021663
Geschäftsführer: George Macdonald
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
take care of the environment - print only if necessary
