Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-11 Thread Norbert Irmer 

If you look at the stacktraces, I sent in my previous mails, you see, that the 
order, in which the
filters are called, is right:

When receiving:

...
at 
org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:310) 
   <---  SslFilter
at org.apache.mina.filter.ssl.SslFilter.messageReceived(SslFilter.java:534)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943)
at 
org.apache.mina.proxy.filter.ProxyFilter.messageReceived(ProxyFilter.java:153)  
   <--- 
ProxyFilter
...


When sending:

---
at org.apache.mina.proxy.filter.ProxyFilter.writeData(ProxyFilter.java:208) 
 <  ProxyFilter
- waiting to lock <0x00076ef8b1d8> (a 
org.apache.mina.proxy.handlers.socks.Socks4LogicHandler)
at org.apache.mina.proxy.filter.ProxyFilter.filterWrite(ProxyFilter.java:192)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:625)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:48)
at 
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:953)
at 
org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:316)
at org.apache.mina.filter.ssl.SslFilter.filterWrite(SslFilter.java:653) 
   < SslFilter



I am only writing the client part of an application.  I communicate with a 
FIX/SSL server behind a SOCKS proxy,
so I cannot change the order in which Ssl encryption and proxy framing must be 
done.

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-11 Thread Emmanuel Lécharny 
Le 11/02/16 10:10, Norbert Irmer a écrit :
> If you look at the stacktraces, I sent in my previous mails, you see, that 
> the order, in which the
> filters are called, is right:
>
> When receiving:
>
> ...
> at 
> org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:310)
> <---  SslFilter
> at org.apache.mina.filter.ssl.SslFilter.messageReceived(SslFilter.java:534)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943)
> at 
> org.apache.mina.proxy.filter.ProxyFilter.messageReceived(ProxyFilter.java:153)
>  
> <--- ProxyFilter
> ...
>
>
> When sending:
>
> ---
> at org.apache.mina.proxy.filter.ProxyFilter.writeData(ProxyFilter.java:208)   
><  ProxyFilter
> - waiting to lock <0x00076ef8b1d8> (a 
> org.apache.mina.proxy.handlers.socks.Socks4LogicHandler)
> at org.apache.mina.proxy.filter.ProxyFilter.filterWrite(ProxyFilter.java:192)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:625)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:48)
> at 
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:953)
> at 
> org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:316)
> at org.apache.mina.filter.ssl.SslFilter.filterWrite(SslFilter.java:653)   
>  < SslFilter
> 
>
>
> I am only writing the client part of an application.  I communicate with a 
> FIX/SSL server behind a SOCKS proxy,
> so I cannot change the order in which Ssl encryption and proxy framing must 
> be done.
Ok, I *think* that the problem is due to the fact that the
flushScheduledEvents() method was not protecting writes *and* reads, so
they can be executed at the same time by two threads. Then what happens
is that both reach the ProxyFilter synchronized section,

FixMessageProcessor takes the SslHandler lock
FixMessageProcessor takes the ProxyFilter lock

and at the same time :

NioProcessor takes the ProxyFilter lock
NioProcessor takes the SslHandler lock

resulting in a Deadlock.


NioProcessor >[ProxyFilter]--->[SslHandler]> IoHandler
NioProcessor <[ProxyFilter]<---[SslHandler]< IoHandler

The new version I have put on http://people.apache.org/~elecharny should
solve the problem : instead of getting the lock whe  we enter in the
flushScheduledEvents() event, we exit immediately, doing nothing but
incrementing the number of scheduled-events, instead of getting the
lock. That means the thread which actually hold this lock will process
all the enqueued messages itself, until the counter get down to 0.

Can you give this version a try ?

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-11 Thread Norbert Irmer 


> The new version I have put on http://people.apache.org/~elecharny should
> solve the problem : instead of getting the lock whe  we enter in the
> flushScheduledEvents() event, we exit immediately, doing nothing but
> incrementing the number of scheduled-events, instead of getting the
> lock. That means the thread which actually hold this lock will process
> all the enqueued messages itself, until the counter get down to 0.
> 
>Can you give this version a try ?


Looks very good, I am running my application now for 20 minutes or so, and still
have no deadlock.

I am using now your snapshot versions without any changes. This is very nice,
now it's possible to send a pull request for proxy support to the QuickFixJ 
people
the next time you do a Mina  release.

Thanks a lot,
Norbert

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-11 Thread Emmanuel Lécharny 
Le 11/02/16 17:38, Norbert Irmer a écrit :
>
>> The new version I have put on http://people.apache.org/~elecharny should
>> solve the problem : instead of getting the lock whe  we enter in the
>> flushScheduledEvents() event, we exit immediately, doing nothing but
>> incrementing the number of scheduled-events, instead of getting the
>> lock. That means the thread which actually hold this lock will process
>> all the enqueued messages itself, until the counter get down to 0.
>>
>> Can you give this version a try ?
>
> Looks very good, I am running my application now for 20 minutes or so, and 
> still
> have no deadlock.
>
> I am using now your snapshot versions without any changes. This is very nice,
> now it's possible to send a pull request for proxy support to the QuickFixJ 
> people
> the next time you do a Mina  release.
>
> Thanks a lot,
> Norbert
>  
>
Glad that seems to be fixed !

Norbert, I have to tell you that I really appreciated your patience, and
I'm really sorry to have tried to deflect the prb for a couple of days.
I really thought it was something else, and it took me some time (and
the price was sleep deprivation :/) to realize that there were a deadlock.

Worst part : this deadlock was already detected months ago by Terence
Marks (https://issues.apache.org/jira/browse/DIRMINA-1019), a patch was
proposed, and I fucked it up thinking it was overdoing. My bad...


I'm going to cut a release asap. Expect it to be ready by the beginning
of next week.

thanks a lot !

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Norbert Irmer 

> seems like you have installed the proxy filter before the ssl filter in
> your chain. Can you do the opposite ?

I don't build the chain manually, I simply use a "ProxyConnector" (from package 
"org.apache.mina.proxy")
to make a proxy connection (this also explains, why I had the 
SslFilter/autoStart problem in my earlier mail).

"ProxyConnector" does the following in "setConnector": 


  private void setConnector(final SocketConnector connector) {
if (connector == null) {
throw new IllegalArgumentException("connector cannot be null");
}

this.connector = connector;
String className = ProxyFilter.class.getName();

// Removes an old ProxyFilter instance from the chain
if (connector.getFilterChain().contains(className)) {
connector.getFilterChain().remove(className);
}

// Insert the ProxyFilter as the first filter in the filter chain 
builder
connector.getFilterChain().addFirst(className, proxyFilter);
}

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Norbert Irmer 

> Ok, but you can insert teh SslFilter *before* the proxyFilter. Once you
> get the ProxyConnector instance, get the connector and do :
 
Well, I am using a patched QuickFixJ version, in which the control flow is a 
bit different.

Here is an excerpt from the code they are using:


  CompositeIoFilterChainBuilder ioFilterChainBuilder = new 
CompositeIoFilterChainBuilder(
userIoFilterChainBuilder);

boolean useProxy = proxyType != null && proxyPort > 0 &&
socketAddresses[0] instanceof InetSocketAddress;


SslFilter sslFilter = null;
if (sslEnabled) {
boolean autoStart = !useProxy;
sslFilter = installSslFilter(ioFilterChainBuilder, autoStart);
}

ioFilterChainBuilder.addLast(FIXProtocolCodecFactory.FILTER_NAME,
new ProtocolCodecFilter(new FIXProtocolCodecFactory()));

if (useProxy) {
SocketConnector socketConnector = new NioSocketConnector();
socketConnector.setFilterChainBuilder(ioFilterChainBuilder);
ioConnector = ProtocolFactory.createIoProxyConnector( 
socketConnector,
(InetSocketAddress) socketAddresses[0], new 
InetSocketAddress(proxyHost,
proxyPort), proxyType, proxyVersion, proxyUser, 
proxyPassword,
proxyDomain, proxyWorkstation);
} else {
ioConnector = 
ProtocolFactory.createIoConnector(socketAddresses[0]);
ioConnector.setFilterChainBuilder(ioFilterChainBuilder);
}

ioHandler = new InitiatorIoHandler(fixSession, sslFilter, 
networkingOptions, eventHandlingStrategy);
}

private SslFilter installSslFilter(CompositeIoFilterChainBuilder 
ioFilterChainBuilder,
   boolean autoStart)
throws GeneralSecurityException {
SslFilter sslFilter = new 
SslFilter(SSLContextFactory.getInstance(keyStoreName,
keyStorePassword.toCharArray()), autoStart);
if (enableProtocole != null)
sslFilter.setEnabledProtocols(enableProtocole);
if (cipherSuites != null)
sslFilter.setEnabledCipherSuites(cipherSuites);
sslFilter.setUseClientMode(true);
ioFilterChainBuilder.addLast(SSLSupport.FILTER_NAME, sslFilter);
return sslFilter;
}


Sure, with a lot of time, I could possibly rewrite the code, to swap the 
positions of the filters in the filterchain.

But  I see little sense in it, since the code is working. The only problem I 
have, is that I get this deadlock after a few minutes or so.

At the moment, as a workaround, I simply disabled the "synchronized(handler) { 
... }" in ProxyFilter.writeData(),
I think for my special case this will work.

I don't think it is possible, that the filterchains are traverted in the same 
order for reading and writing.

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Emmanuel Lécharny 
Le 10/02/16 16:37, Norbert Irmer a écrit :
>> seems like you have installed the proxy filter before the ssl filter in
>> your chain. Can you do the opposite ?
> I don't build the chain manually, I simply use a "ProxyConnector" (from 
> package "org.apache.mina.proxy")
> to make a proxy connection (this also explains, why I had the 
> SslFilter/autoStart problem in my earlier mail).
>
> "ProxyConnector" does the following in "setConnector": 
>
>
>   private void setConnector(final SocketConnector connector) {
> if (connector == null) {
> throw new IllegalArgumentException("connector cannot be null");
> }
>
> this.connector = connector;
> String className = ProxyFilter.class.getName();
>
> // Removes an old ProxyFilter instance from the chain
> if (connector.getFilterChain().contains(className)) {
> connector.getFilterChain().remove(className);
> }
>
> // Insert the ProxyFilter as the first filter in the filter chain 
> builder
> connector.getFilterChain().addFirst(className, proxyFilter);
> }
Ok, but you can insert teh SslFilter *before* the proxyFilter. Once you
get the ProxyConnector instance, get the connector and do :


SslFilter sslFilter = new SslFilter(BogusSslContextFactory
.getInstance(true));
connector.getFilterChain().addFirst( "MySslFilter", sslFilter );

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Emmanuel Lécharny 
Le 10/02/16 18:16, Norbert Irmer a écrit :
>> Ok, but you can insert teh SslFilter *before* the proxyFilter. Once you
>> get the ProxyConnector instance, get the connector and do :
>  
> Well, I am using a patched QuickFixJ version, in which the control flow is a 
> bit different.
>
> Here is an excerpt from the code they are using:
>
>
>   CompositeIoFilterChainBuilder ioFilterChainBuilder = new 
> CompositeIoFilterChainBuilder(
> userIoFilterChainBuilder);
>
> boolean useProxy = proxyType != null && proxyPort > 0 &&
> socketAddresses[0] instanceof 
> InetSocketAddress;
>
>
> SslFilter sslFilter = null;
> if (sslEnabled) {
> boolean autoStart = !useProxy;
> sslFilter = installSslFilter(ioFilterChainBuilder, autoStart);
> }
>
> ioFilterChainBuilder.addLast(FIXProtocolCodecFactory.FILTER_NAME,
> new ProtocolCodecFilter(new FIXProtocolCodecFactory()));
>
> if (useProxy) {
> SocketConnector socketConnector = new NioSocketConnector();
> socketConnector.setFilterChainBuilder(ioFilterChainBuilder);
> ioConnector = ProtocolFactory.createIoProxyConnector( 
> socketConnector,
> (InetSocketAddress) socketAddresses[0], new 
> InetSocketAddress(proxyHost,
> proxyPort), proxyType, proxyVersion, 
> proxyUser, proxyPassword,
> proxyDomain, proxyWorkstation);
> } else {
> ioConnector = 
> ProtocolFactory.createIoConnector(socketAddresses[0]);
> ioConnector.setFilterChainBuilder(ioFilterChainBuilder);
> }
>
> ioHandler = new InitiatorIoHandler(fixSession, sslFilter, 
> networkingOptions, eventHandlingStrategy);
> }
>
> private SslFilter installSslFilter(CompositeIoFilterChainBuilder 
> ioFilterChainBuilder,
>boolean autoStart)
> throws GeneralSecurityException {
> SslFilter sslFilter = new 
> SslFilter(SSLContextFactory.getInstance(keyStoreName,
> keyStorePassword.toCharArray()), autoStart);
> if (enableProtocole != null)
> sslFilter.setEnabledProtocols(enableProtocole);
> if (cipherSuites != null)
> sslFilter.setEnabledCipherSuites(cipherSuites);
> sslFilter.setUseClientMode(true);
> ioFilterChainBuilder.addLast(SSLSupport.FILTER_NAME, sslFilter);
> return sslFilter;
> }
>
>
> Sure, with a lot of time, I could possibly rewrite the code, to swap the 
> positions of the filters in the filterchain.
>
> But  I see little sense in it, since the code is working. The only problem I 
> have, is that I get this deadlock after a few minutes or so.
How can you say the code is working when you get a deadlock ?

As Jon said, having the ProcyFilter *before* the SSLFilter in the chain
is not the right thing. The reason being that you may perfectly get
fragmented TCP buffers, and the proxy might not be able to deal with
de-fragmentation when it comes to encrypted data.

I suggest that you patch teh code to first call the installSslFilter
method after the proxy has been created, and in the
installSslFiltermethod, use addFirst() instead of addLast().

Otherwise, you can also check with the latest trunk revision I built
today to see if it's any better (on http://people.apache.org/~elecharny)

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Jon V. 
I would have thought the reason SSL Filter needs to be before Proxy is
because the SSL handshaking from the client side would get proxied away
from the SSL Filter.

On Wed, Feb 10, 2016 at 12:57 PM, Emmanuel Lécharny 
wrote:

> Le 10/02/16 18:16, Norbert Irmer a écrit :
> >> Ok, but you can insert teh SslFilter *before* the proxyFilter. Once you
> >> get the ProxyConnector instance, get the connector and do :
> >
> > Well, I am using a patched QuickFixJ version, in which the control flow
> is a bit different.
> >
> > Here is an excerpt from the code they are using:
> >
> >
> >   CompositeIoFilterChainBuilder ioFilterChainBuilder = new
> CompositeIoFilterChainBuilder(
> > userIoFilterChainBuilder);
> >
> > boolean useProxy = proxyType != null && proxyPort > 0 &&
> > socketAddresses[0] instanceof
> InetSocketAddress;
> >
> >
> > SslFilter sslFilter = null;
> > if (sslEnabled) {
> > boolean autoStart = !useProxy;
> > sslFilter = installSslFilter(ioFilterChainBuilder,
> autoStart);
> > }
> >
> >
>  ioFilterChainBuilder.addLast(FIXProtocolCodecFactory.FILTER_NAME,
> > new ProtocolCodecFilter(new
> FIXProtocolCodecFactory()));
> >
> > if (useProxy) {
> > SocketConnector socketConnector = new
> NioSocketConnector();
> >
>  socketConnector.setFilterChainBuilder(ioFilterChainBuilder);
> > ioConnector = ProtocolFactory.createIoProxyConnector(
> socketConnector,
> > (InetSocketAddress) socketAddresses[0], new
> InetSocketAddress(proxyHost,
> > proxyPort), proxyType, proxyVersion,
> proxyUser, proxyPassword,
> > proxyDomain, proxyWorkstation);
> > } else {
> > ioConnector =
> ProtocolFactory.createIoConnector(socketAddresses[0]);
> > ioConnector.setFilterChainBuilder(ioFilterChainBuilder);
> > }
> >
> > ioHandler = new InitiatorIoHandler(fixSession, sslFilter,
> networkingOptions, eventHandlingStrategy);
> > }
> >
> > private SslFilter installSslFilter(CompositeIoFilterChainBuilder
> ioFilterChainBuilder,
> >boolean autoStart)
> > throws GeneralSecurityException {
> > SslFilter sslFilter = new
> SslFilter(SSLContextFactory.getInstance(keyStoreName,
> > keyStorePassword.toCharArray()), autoStart);
> > if (enableProtocole != null)
> > sslFilter.setEnabledProtocols(enableProtocole);
> > if (cipherSuites != null)
> > sslFilter.setEnabledCipherSuites(cipherSuites);
> > sslFilter.setUseClientMode(true);
> > ioFilterChainBuilder.addLast(SSLSupport.FILTER_NAME,
> sslFilter);
> > return sslFilter;
> > }
> >
> >
> > Sure, with a lot of time, I could possibly rewrite the code, to swap the
> positions of the filters in the filterchain.
> >
> > But  I see little sense in it, since the code is working. The only
> problem I have, is that I get this deadlock after a few minutes or so.
> How can you say the code is working when you get a deadlock ?
>
> As Jon said, having the ProcyFilter *before* the SSLFilter in the chain
> is not the right thing. The reason being that you may perfectly get
> fragmented TCP buffers, and the proxy might not be able to deal with
> de-fragmentation when it comes to encrypted data.
>
> I suggest that you patch teh code to first call the installSslFilter
> method after the proxy has been created, and in the
> installSslFiltermethod, use addFirst() instead of addLast().
>
> Otherwise, you can also check with the latest trunk revision I built
> today to see if it's any better (on http://people.apache.org/~elecharny)
>
>

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Emmanuel Lécharny 
Le 10/02/16 10:08, Norbert Irmer a écrit :
> I attached a stacktrace of the two threads, which are in a deadlock.
>
> Jon maybe right, when saying the write stack is threadsafe, the problem is, 
> that simultaneously
> a read operation is going on, and both stacks use the same filters.

The stakcs you provided are useless. What we really need here is the
full stack dump.

Also note that the filter is instanciated as many times as there are
sessions, so the real problem is when a session is processed by two
different threads. That might make sense in some use case, but we have
to guarantee that the encryption and decryption of messages are done
fully before dealing with the next message. This is the reason we have a
lock in the SslHandler.

That being said, there *is* something wrong and we need to figure out
what it is and how to get it fixed.

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Norbert Irmer 
I attached a stacktrace of the two threads, which are in a deadlock.

Jon maybe right, when saying the write stack is threadsafe, the problem is, 
that simultaneously
a read operation is going on, and both stacks use the same filters.

Thread "NioProcessor-2": 
 
flushScheduledEvents():308, SslHandler {org.apache.mina.filter.ssl}
messageReceived():534, SslFilter {org.apache.mina.filter.ssl}
callNextMessageReceived():542, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
access$1300():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
messageReceived():943, DefaultIoFilterChain$EntryImpl$1 
{org.apache.mina.core.filterchain}
messageReceived():153, ProxyFilter {org.apache.mina.proxy.filter}
callNextMessageReceived():542, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
access$1300():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
messageReceived():943, DefaultIoFilterChain$EntryImpl$1 
{org.apache.mina.core.filterchain}
messageReceived():109, IoFilterAdapter {org.apache.mina.core.filterchain}
callNextMessageReceived():542, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
fireMessageReceived():535, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
read():684, AbstractPollingIoProcessor {org.apache.mina.core.polling}
process():638, AbstractPollingIoProcessor {org.apache.mina.core.polling}
process():627, AbstractPollingIoProcessor {org.apache.mina.core.polling}
access$600():68, AbstractPollingIoProcessor {org.apache.mina.core.polling}
run():1096, AbstractPollingIoProcessor$Processor {org.apache.mina.core.polling}
run():64, NamePreservingRunnable {org.apache.mina.util}
runWorker():1142, ThreadPoolExecutor {java.util.concurrent}
run():617, ThreadPoolExecutor$Worker {java.util.concurrent}
run():745, Thread {java.lang}


Thread "FixMessageProcessor(0)::ABFX":

writeData():209, ProxyFilter {org.apache.mina.proxy.filter}
filterWrite():192, ProxyFilter {org.apache.mina.proxy.filter}
callPreviousFilterWrite():625, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1 
{org.apache.mina.core.filterchain}
flushScheduledEvents():316, SslHandler {org.apache.mina.filter.ssl}
filterWrite():653, SslFilter {org.apache.mina.filter.ssl}
callPreviousFilterWrite():625, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1 
{org.apache.mina.core.filterchain}
filterWrite():321, ProtocolCodecFilter {org.apache.mina.filter.codec}
callPreviousFilterWrite():625, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1 
{org.apache.mina.core.filterchain}
filterWrite():877, DefaultIoFilterChain$TailFilter 
{org.apache.mina.core.filterchain}
callPreviousFilterWrite():625, DefaultIoFilterChain 
{org.apache.mina.core.filterchain}
fireFilterWrite():618, DefaultIoFilterChain {org.apache.mina.core.filterchain}
write():530, AbstractIoSession {org.apache.mina.core.session}
write():474, AbstractIoSession {org.apache.mina.core.session}
send():51, IoSessionResponder {quickfix.mina}
send():2534, Session {quickfix}
sendRaw():2466, Session {quickfix}
send():2519, Session {quickfix}
sendToTarget():635, Session {quickfix}
sendMessageToTarget():529, FixApplication {de.digitec.d3.gateway.fix}
subscribe():52, FixProducerHandle {de.digitec.d3.gateway.fix}
subscribe():134, FixApplication {de.digitec.d3.gateway.fix}
processMessage():288, FixMessageProcessor {de.digitec.d3.gateway.fix}
run():245, FixMessageProcessor {de.digitec.d3.gateway.fix}
run():878, FixApplication$MessageProcessorThread {de.digitec.d3.gateway.fix}






DIGITEC GmbH
Johannes-Brahms-Platz 1
20355 Hamburg

Tel +49 40 23776-147
Fax +49 40 23776-200
www.DIGITEC.de

___

DIGITEC GmbH EDV-Beratung, Soft- und Hardware
Sitz Hamburg  -  HRB 288 18 Amtsgericht Hamburg
Geschäftsführer: Hans-Joachim Hänel, Hans-Jürgen Joost
Finanzamt Hamburg-Mitte, Steuer-Nr. 48/715/01586  -  VAT-Nr. DE 118646977
WICHTIGER HINWEIS
Die Kommunikation mit DIGITEC GmbH per Email dient nur dem Zweck der schnellen 
Information.
Rechtsverbindliche Absprachen kommen über dieses Medium nicht zustande !
IMPORTANT NOTICE
Communication with digitec GmbH via email is for quick information purposes 
only.
This medium is not to be used for the exchange of legally binding statements !

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Norbert Irmer 
Hello,

I did a "threaddump" with visualvm in the deadlock situation (not sure if this 
is what you wanted, but I don't see
anything else), it shows the same deadlock situation (at the bottom of this 
mail), as in my previous mail.


2016-02-10 13:04:58
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode):

"Thread-7" #29 daemon prio=5 os_prio=0 tid=0x1ed49000 nid=0xfb0 
runnable [0x1f66e000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
- locked <0x00076ee06830> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.skip(BufferedReader.java:411)
- locked <0x00076ee06830> (a java.io.InputStreamReader)
at 
de.digitec.d3.pricing.streaming.impl.StreamingChannelImpl.readMessage(StreamingChannelImpl.java:331)
at de.digitec.d3.pricing.streaming.impl.Session.run(Session.java:116)
- locked <0x00076ee068c8> (a java.lang.Object)

   Locked ownable synchronizers:
- None

"SiDispatcher_ReadLoop" #28 prio=5 os_prio=0 tid=0x1ed48800 nid=0x17fc 
waiting on condition [0x221de000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00076f04c960> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
de.digitec.d3.pricing.streaming.impl.SessionManagerAbstractImpl.readMessage(SessionManagerAbstractImpl.java:49)
at 
de.digitec.d3.pricing.streaming.impl.SiDispatcher.run(SiDispatcher.java:455)
at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
- None

"Thread-6" #27 daemon prio=5 os_prio=0 tid=0x1ed47800 nid=0x1674 
runnable [0x2202f000]
   java.lang.Thread.State: RUNNABLE
at java.net.DualStackPlainSocketImpl.accept0(Native Method)
at 
java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
at 
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
- locked <0x00076ee102c8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:545)
at java.net.ServerSocket.accept(ServerSocket.java:513)
at 
de.digitec.d3.pricing.streaming.impl.SocketAcceptor.accept(SocketAcceptor.java:152)
at 
de.digitec.d3.pricing.streaming.impl.SocketAcceptor.run(SocketAcceptor.java:128)

   Locked ownable synchronizers:
- None

"SessionManagerStreamingImpl_Timer" #26 daemon prio=5 os_prio=0 
tid=0x1ed47000 nid=0x1bbc in Object.wait() [0x21d9f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00076ee18180> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)

   Locked ownable synchronizers:
- None

"SiDispatcher_Timer" #25 prio=5 os_prio=0 tid=0x1ed46000 nid=0x12d8 in 
Object.wait() [0x2030f000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.util.TimerThread.mainLoop(Timer.java:526)
- locked <0x00076ee18350> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)

   Locked ownable synchronizers:
- None

"RMI TCP Connection(2)-172.16.125.86" #24 daemon prio=5 os_prio=0 
tid=0x1ed45800 nid=0x1704 runnable [0x218ad000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00076ee12618> (a java.io.BufferedInputStream)
at

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Emmanuel Lécharny 


Le 10/02/16 13:11, Norbert Irmer a écrit :
> Hello,
>
> I did a "threaddump" with visualvm in the deadlock situation (not sure if 
> this is what you wanted, but I don't see
> anything else), it shows the same deadlock situation (at the bottom of this 
> mail), as in my previous mail.

Thanks !


seems like you have installed the proxy filter before the ssl filter in
your chain. Can you do the opposite ?

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-10 Thread Emmanuel Lécharny 
Le 10/02/16 22:11, Jon V. a écrit :
> I would have thought the reason SSL Filter needs to be before Proxy is
> because the SSL handshaking from the client side would get proxied away
> from the SSL Filter.

That too. But all in all, the handshake is just an exchange of data, as
any other data beinge exchanged later on. It can also be split in
pieces, the wame way...

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-09 Thread Jonathan Valliere 
Obviously something is wrong with the thread-safe operation the write stack.  
If it was thread safe then there is no need to ever have locks in the filter 
chain unless you have an executor filter. 

1027 exposes a bug in the lock routines of IoSession

Sent from my iPhone

> On Feb 9, 2016, at 12:48 PM, Emmanuel Lécharny  wrote:
> 
> Le 09/02/16 18:22, Jon V. a écrit :
>> Its my understanding that Read and Write operations are thread-safe and
>> should never cause deadlocks. (not sure why there are locks in those
>> filters)
>> 
>> Also, I don’t think SSL should be after Proxy.  You cannot initiate an SSL
>> session if you proxy the setup routine.  What is the goal in this setup?
>> 
>> Please send us the deadlock thread trace.(You can get it from VisualVM)
> 
> I think this is a pb that is already exposed in DIRMINA-1027.
> 
> I still have to find a couple of hours to analyse the code and find a
> solution.
>

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-09 Thread Jon V. 
Its my understanding that Read and Write operations are thread-safe and
should never cause deadlocks. (not sure why there are locks in those
filters)

Also, I don’t think SSL should be after Proxy.  You cannot initiate an SSL
session if you proxy the setup routine.  What is the goal in this setup?

Please send us the deadlock thread trace.(You can get it from VisualVM)

On Tue, Feb 9, 2016 at 11:59 AM, Norbert Irmer  wrote:

> Hello,
>
> I detected a deadlock when using Proxies and Ssl together.
>
> The SslFilter and the ProxyFilzter both have synchronized blocks, and when
> sending
> and receiving messages you get deadlocks.
>
> The SslFilter hangs in "sslLock.lock()" in
> "SslHandler.flushScheduledEvents()",
> while the ProxyFilter hangs in "synchronized(handler)" in
> "ProxyFilter.writeData".
>
> That sending and receiving messages with both filters simultaneously leads
> to a deadlock
> is clear, since both filters have locks and data flow direction is
> reversed when reading and writing:
>
> Read direction:   Socket  --> ProxyFilter   ---> SslFilter  -->
> FixProtocol   --> Application
> Write direction:  Socket <-- ProxyFilter  <---  SslFilter  <--
> FixProtocol  <-- Application
>
>
>
>

Re: Deadlock when using SslFilter and ProxyFilter together

2016-02-09 Thread Emmanuel Lécharny 
Le 09/02/16 18:22, Jon V. a écrit :
> Its my understanding that Read and Write operations are thread-safe and
> should never cause deadlocks. (not sure why there are locks in those
> filters)
>
> Also, I don’t think SSL should be after Proxy.  You cannot initiate an SSL
> session if you proxy the setup routine.  What is the goal in this setup?
>
> Please send us the deadlock thread trace.(You can get it from VisualVM)

I think this is a pb that is already exposed in DIRMINA-1027.

I still have to find a couple of hours to analyse the code and find a
solution.