Re: [twsocket] SSL problem (server stops receiving data)
Thanks for the update Angus! Merijn On 2/1/2016 20:42, Angus Robertson - Magenta Systems Ltd wrote: We've hired someone to look into this issue, this resulted in finding the probable cause of this issue, and a fix. It would be great if someone could sort of validate the conclusion and fix. If all is OK, how to get these changes into the ICS codebase? Your developer updated a six month old WSocket, so I added your changes to the latest version. Building your test application with the old v8.21 causes SSL to die within a second, with the new V8.22 it works OK creating a gigabyte SSL log file with lots of successful requests. I don't understand why your fix works, it seems to execute the same code as before, just in a different place, but clearly there was an SSL problem before in your particular circumstance with overlapping requests which is now fixed. I've rebuilt my web application server and put it on my public web site, where it will get a few hundred thousand requests quickly, although I've not knowingly had any problems before. If there are no new problems, I'll updated SVN in a few days. My test page showing ICS versions is at: https://www.telecom-tariffs.co.uk/serverinfo.htm Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
> We've hired someone to look into this issue, this resulted in > finding the probable cause of this issue, and a fix. > It would be great if someone could sort of validate the > conclusion and fix. > If all is OK, how to get these changes into the ICS codebase? Your developer updated a six month old WSocket, so I added your changes to the latest version. Building your test application with the old v8.21 causes SSL to die within a second, with the new V8.22 it works OK creating a gigabyte SSL log file with lots of successful requests. I don't understand why your fix works, it seems to execute the same code as before, just in a different place, but clearly there was an SSL problem before in your particular circumstance with overlapping requests which is now fixed. I've rebuilt my web application server and put it on my public web site, where it will get a few hundred thousand requests quickly, although I've not knowingly had any problems before. If there are no new problems, I'll updated SVN in a few days. My test page showing ICS versions is at: https://www.telecom-tariffs.co.uk/serverinfo.htm Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
> We've hired someone to look into this issue, this resulted in > finding the probable cause of this issue, and a fix. > It would be great if someone could sort of validate the > conclusion and fix. > If all is OK, how to get these changes into the ICS codebase? Zip any ICS units and test applications you created and email them to my email address. If the changes are 'safe', I'll update SVN. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
Hi All, We've hired someone to look into this issue, this resulted in finding the probable cause of this issue, and a fix. It would be great if someone could sort of validate the conclusion and fix. If all is OK, how to get these changes into the ICS codebase? Thanks, Merijn On 12/8/2015 17:42, Angus Robertson - Magenta Systems Ltd wrote: I really hope someone can take a look at this, maybe confirm if the problem is reproducible and see if I'm doing something wrong or this is indeed a problem inside ICS / OpenSSL. Sorry, I will not have any time to investigate for several weeks, or longer. Maybe another volunteer here has more free time than I can spare. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
> I really hope someone can take a look at this, maybe confirm if the > problem is reproducible and see if I'm doing something wrong or this is > indeed a problem inside ICS / OpenSSL. Sorry, I will not have any time to investigate for several weeks, or longer. Maybe another volunteer here has more free time than I can spare. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
Hi Angus, I only mentioned the renegotiation because when I found references to that when trying to solve this problem, I'm not sure if this is actually what causes it. I'm not initiating any renegotiation myself, nor have I seen anything about that in the logs. Meanwhile, I haven't been able to solve this yet, however, I did found that the problem can be quite easily reproduced using the OverbyteIcsSimpleSslCli and OverbyteIcsSimpleSslServer demo projects, and their certificates, provided in the ICS download. I've made some little changes on both projects to get the request / reply game running, besides this, the client has a timer on which it sends 'spontaneous' data (that means not initiated by the request / reply mechanics). The more I dig into this, the more it seems that it's either a bug on OpenSSL, or a bug in the ICS implementation / ICS does something which is not allowed by OpenSSL. I really can't imagine it's a bug in OpenSSL, so it's pointing mainly towards user error or problems in ICS itself. I'm really hoping I'm doing something terribly wrong, but at the moment it looks like using ICS with OpenSSL for a connection where spontaneous data is being sent will not give a stable result :( I've uploaded the altered ICS demo projects here: http://www.xs4all.nl/~bosma/OverbyteIcsSimple.zip I really hope someone can take a look at this, maybe confirm if the problem is reproducible and see if I'm doing something wrong or this is indeed a problem inside ICS / OpenSSL. To reproduce with these test projects do as follows: - start server app - press 'start' button on server app - start client app - press 'connect' button on client app - as soon as the SSL authentication is done, you will see the client sending frequent keep alives and the server receiving them (each 500 ms) - press the 'start' button on the client, this will make the client send a request, and the server sent 9000 bytes of data back - after a short while, you will see the server stops receiving data, the request / reply routine stops working (the server doens't receive the requests anymore), you will see the client still sending keep alives, but the server does not receive them anymore. - in this situation, the server will never receive data on that socket anymore. Hoping for a push into the right direction. thanks in advance, Merijn On 25/11/2015 17:56, Angus Robertson - Magenta Systems Ltd wrote: In short, it seems that OpenSSL can get confused when application data is sent while it is doing renegotiation itself, if I understood correctly. Could this be the problem we are experiencing, or does the ICS implementation around OpenSSL take this into account? Why would you application be doing renegotiation? Did you see any of that in the logs. As far as I'm aware, the ICS OpenSSL implementation is full duplex, as is TCP/IP. But most protocols are essentially half duplex, unless streaming lining is used. But I did not write ICS OpenSSL implementation, and try to avoid the deep complexities of it, except for the parts I keep updating to improve it. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
> In short, it seems that OpenSSL can get confused when application > data is sent while it is doing renegotiation itself, if I > understood correctly. > Could this be the problem we are experiencing, or does the ICS > implementation around OpenSSL take this into account? Why would you application be doing renegotiation? Did you see any of that in the logs. As far as I'm aware, the ICS OpenSSL implementation is full duplex, as is TCP/IP. But most protocols are essentially half duplex, unless streaming lining is used. But I did not write ICS OpenSSL implementation, and try to avoid the deep complexities of it, except for the parts I keep updating to improve it. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
It seems that this issue is triggered when we sent and receive data at the same time. What happens is that we have a client and a server doing answer request, either side can also send data based on a timer, sending that data can trigger this issue. Googling for "OpenSSL full duplex" or similar, gives quite a bit of hits on people experiencing similar issues, like these: http://www.scriptscoop.net/t/0f9aca924ddc/ssl-renegotiation-with-full-duplex-socket-communication.html http://openssl.6102.n7.nabble.com/Allowing-fullduplex-in-SSL-td46443.html https://github.com/FreeRDP/FreeRDP/issues/2497 In short, it seems that OpenSSL can get confused when application data is sent while it is doing renegotiation itself, if I understood correctly. Could this be the problem we are experiencing, or does the ICS implementation around OpenSSL take this into account? Thanks, Merijn On 24/11/2015 18:22, Merijn Bosma wrote: Hi Angus, I agree with what you are saying, but in this case this is not the problem. The only reason this app works like this, is because this seems to be the easiest way to reproduce this same issue which happens in a larger app, which does use a FIFO etc. Log clearly shows that the two random numbers are being received separately on the server side. Merijn On 24/11/2015 17:23, Angus Robertson - Magenta Systems Ltd wrote: The problem is triggered, when we do two times PostMessage(WM_USER) in the OnSslHandshakeDone event, expected behavior would be that the client sends a random number twice, server receives the first, sends x bytes and term char, client receives it, sends next random number (3th), server might be handling the 2nd number, etc. I'd expect the client to send a single TCP/IP packet with both random numbers in it, and for the server to receive both together. So does the server have a FIFO buffer to store the second number for processing later, that take priority over anything received. I had this problem a long time ago with a simple packet protocol. The difference between SSL and non-SSL might be packets being combined. Try putting a delay in when sending, so there is always a two or longer second gap and see if the problem goes away. But the real solution is the FIFO buffer. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
Hi Angus, I agree with what you are saying, but in this case this is not the problem. The only reason this app works like this, is because this seems to be the easiest way to reproduce this same issue which happens in a larger app, which does use a FIFO etc. Log clearly shows that the two random numbers are being received separately on the server side. Merijn On 24/11/2015 17:23, Angus Robertson - Magenta Systems Ltd wrote: The problem is triggered, when we do two times PostMessage(WM_USER) in the OnSslHandshakeDone event, expected behavior would be that the client sends a random number twice, server receives the first, sends x bytes and term char, client receives it, sends next random number (3th), server might be handling the 2nd number, etc. I'd expect the client to send a single TCP/IP packet with both random numbers in it, and for the server to receive both together. So does the server have a FIFO buffer to store the second number for processing later, that take priority over anything received. I had this problem a long time ago with a simple packet protocol. The difference between SSL and non-SSL might be packets being combined. Try putting a delay in when sending, so there is always a two or longer second gap and see if the problem goes away. But the real solution is the FIFO buffer. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL problem (server stops receiving data)
> The problem is triggered, when we do two times PostMessage(WM_USER) > in the OnSslHandshakeDone event, expected behavior would be that > the client sends a random number twice, server receives the first, > sends x bytes and term char, client receives it, sends next random > number (3th), server might be handling the 2nd number, etc. I'd expect the client to send a single TCP/IP packet with both random numbers in it, and for the server to receive both together. So does the server have a FIFO buffer to store the second number for processing later, that take priority over anything received. I had this problem a long time ago with a simple packet protocol. The difference between SSL and non-SSL might be packets being combined. Try putting a delay in when sending, so there is always a two or longer second gap and see if the problem goes away. But the real solution is the FIFO buffer. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL Problem
Hi Angus, On Thu, 5 Mar 2015 07:57 + (GMT Standard Time), you wrote: > > > My ICS has revised date: Sept 3, 2014. It's a nightly snapshot. I > > have OpenSSL 1.0.1k (compiled by you). > > You may have have them, but it's unlikely you are using them together since > 1.0.1k was only released on 19th January 2015 and needs a nightly snapshot > dated then or later to install it. ICS does not load newer versions of > OpenSSL > that have not been tested. When you tested it and made it available, I added the version myself. I'm a programmer :) > > I have an SMTP client and an HTTP client. > > SSL client have much less control over ciphers than servers, essentially only > flags like sslOpt_NO_TLSv1, sslOpt_NO_SSLv2, sslOpt_NO_SSLv3 to refuse old > ciphers. SslVersionMethod is very crude and does not support TLS 1.2, so you > have to leave it as sslV23_CLIENT. Ok, that explains the sslV23_Client. Thanks. > If you use a specific CipherList with a client, you risk being unable to > access > a server that does not match it, maybe not today, but probably tomorrow when > the server is hardened. SSL servers need to be updated frequently to counter > new threats. > > > Can you tell me why I get the 'SSL3_CLIENT_HELLO:no ciphers > > available' fatal error I have a USENET news > > reader program that uses an indy nntp client and the suite works > > fine with it. > > Ciphers are primarily chosen by the server, so unless you are using the ICS > SMTP and HTTP client to talk to an NTTP news server, what Indy supports is > irrelevant. Good point. As for the 'no ciphers available', I'll assume it's another crude implementation. I can't see any other reason. Thank you for your assistance. I now have my clients using TLS 1.2. That was my problem and now it's fixed. George -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL Problem
> My ICS has revised date: Sept 3, 2014. It's a nightly snapshot. I > have OpenSSL 1.0.1k (compiled by you). You may have have them, but it's unlikely you are using them together since 1.0.1k was only released on 19th January 2015 and needs a nightly snapshot dated then or later to install it. ICS does not load newer versions of OpenSSL that have not been tested. > I have an SMTP client and an HTTP client. SSL client have much less control over ciphers than servers, essentially only flags like sslOpt_NO_TLSv1, sslOpt_NO_SSLv2, sslOpt_NO_SSLv3 to refuse old ciphers. SslVersionMethod is very crude and does not support TLS 1.2, so you have to leave it as sslV23_CLIENT. If you use a specific CipherList with a client, you risk being unable to access a server that does not match it, maybe not today, but probably tomorrow when the server is hardened. SSL servers need to be updated frequently to counter new threats. > Can you tell me why I get the 'SSL3_CLIENT_HELLO:no ciphers > available' fatal error I have a USENET news > reader program that uses an indy nntp client and the suite works > fine with it. Ciphers are primarily chosen by the server, so unless you are using the ICS SMTP and HTTP client to talk to an NTTP news server, what Indy supports is irrelevant. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL Problem
On Tue, 3 Mar 2015 08:43 + (GMT Standard Time), you wrote: > > > I have a high security email program that I'm trying to correct for > > POODLE in. It also uses https. > > Which version of ICS TWSocket are you using, and which OpenSSL version? 1.0.1j > fixed Poodle. Is this a client or server? My ICS has revised date: Sept 3, 2014. It's a nightly snapshot. I have OpenSSL 1.0.1k (compiled by you). I have an SMTP client and an HTTP client. > A client has much less control over ciphers than a server, the latest ICS V8 > provides several levels of Ciphers used by Mozilla with > sslCiphersMozillaSrvHigh being the best. > > This was all discussed when I explained how to stop Poodle in a mailing list > post on 20 October 2014. > Connecting to Gmail, I get excellent ciphers: > > SSL Connected OK with TLSv1.2, cipher ECDHE-RSA-AES128-GCM-SHA256, key > exchange > ECDH, encryption AESGCM(128), message authentication AEAD Well, I tried the mozilla ciphersuite and it didn't work. Then when I set SslVersionMethod back from sslTLS_v1 to sslV23 it worked! Go figure. sslTLS_V1 was causing it to use only TLS 1.0. Now I'm getting TLS1.2. Great! Can you tell me why I get the 'SSL3_CLIENT_HELLO:no ciphers available' fatal error (as mentioned in my first message), when using the cipher suite ALL:!ADH:!MD5:!SSLv3:+TLSv1.2:@STRENGTH. I have a USENET news reader program that uses an indy nntp client and the suite works fine with it. Thanks, George -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL Problem
> I have a high security email program that I'm trying to correct for > POODLE in. It also uses https. Which version of ICS TWSocket are you using, and which OpenSSL version? 1.0.1j fixed Poodle. Is this a client or server? A client has much less control over ciphers than a server, the latest ICS V8 provides several levels of Ciphers used by Mozilla with sslCiphersMozillaSrvHigh being the best. This was all discussed when I explained how to stop Poodle in a mailing list post on 20 October 2014. Connecting to Gmail, I get excellent ciphers: SSL Connected OK with TLSv1.2, cipher ECDHE-RSA-AES128-GCM-SHA256, key exchange ECDH, encryption AESGCM(128), message authentication AEAD Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
