Hello all,
I'm struggling with an issue which occurs when we use OpenSSL on a
normal socket (TSslWSocketClient connecting to TSslWSocketServer).
If put very short, the problem means that at some point, data sent from
the client to the server is never received by the server. If this state
occurs, any data sent from client to server simply is not received
anymore for that connection, data sent from the server to the client
still is received by the client. No error or other events on the client
when we are in this state, the connection still seems to be up.
I've been working to make the problem as small as possible, and I end up
with a little application which reproduces this issue 100% of the time.
If we try the same application, just without using SSL (TWSocketClient
connecting to TCustomMultiListenWSocketServer) the problem cannot be
reproduced, hence the problem seems to lie around OpenSSL. This
application is made using Delphi XE7 with the latest stable ICS version
(8.18), using the latest stable OpenSSL version (1.0.2d).
The application runs both the client and the server, and functions as
follows.
1) When the client gets an OnSslHandshakeDone event, it will do a
PostMessage(WM_USER) to itself.
2) When this WM_USER is handled, the client will sent a random number to
the server, where (8192 <= number < 9215)
3) When the server receives this number, it will generate a block of
data of the length of that number, fill it with some random ansi data,
and send it back to the client (with a term character at the end).
3) The client will receive the data and buffer it, when it receives the
term character it throws away all data and do a PostMessage(WM_USER) to
itself, back to step 2
This game of answer and reply works ok, and looks like this in log:
16:13:01.122 client application connected (0)
16:13:01.268 server application client connected (0)
16:13:01.357 server application SSL handshake done (0)
16:13:01.359 client application SSL handshake done (0)
16:13:01.359 client application sending request (8967)
16:13:01.359 server application rx request for 8967 bytes
16:13:01.360 client application rx 4096 bytes of data
16:13:01.361 client application rx 4096 bytes of data
16:13:01.361 client application rx 776 bytes of data
16:13:01.361 client application rx complete block (8967 bytes)
16:13:01.361 client application sending request (9190)
16:13:01.361 server application rx request for 9190 bytes
16:13:01.362 client application rx 4096 bytes of data
16:13:01.362 client application rx 4096 bytes of data
16:13:01.363 client application rx 999 bytes of data
16:13:01.363 client application rx complete block (9190 bytes)
...etc
The problem is triggered, when we do two times PostMessage(WM_USER) in
the OnSslHandshakeDone event, expected behavior would be that the client
sends a random number twice, server receives the first, sends x bytes
and term char, client receives it, sends next random number (3th),
server might be handling the 2nd number, etc.
However, what happens is this:
- client sends random number (#1) and another random number (#2)
- server receives random number (#1) and sends x bytes + term char back
to client
- client receives x bytes from random number #1 + term char from server
and sends a new random number (#3)
- server receives random number (#2) and sends x bytes + term char back
to client
- client receives x bytes from random number #2 + term char from server
and sends a new random number (#4)
And there it stops, random number #3 and #4 are never received by the
server, this behavior is found here exactly the same every time run.
The application produces log like above, but also all log which is
gathered by ICS logger using [loSslErr, loSslInfo, loSslDump] for both
client and server. If you dig into that log it always ends with the last
random number sent by the client being picked up by SSL, it looks like
it's being sent (Do_FD_WRITE), some SSL stuff, everything looks normal,
it's just never received by the server. Snippet of long when it goes wrong:
16:32:02.516 client application sending request (8272)
16:32:02.516 client loSslInfo 02424090 PutDataInSslBuffer 512 len
4 [609]
16:32:02.516 client loSslInfo 02424090 SslTryToSend 512
16:32:02.516 client loSslInfo 02424090 BIO_write(sslbio,
0x2442B80, 4) = 4 [610]
16:32:02.516 client loSslInfo 02424090 BIO_ctrl(sslbio,
BIO_CTRL_FLUSH, 0, 0x0) = 1 [611]
16:32:02.516 client loSslInfo 02424090 TriggerEvents 512 SslState:
SSL_ST_OK // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1
MaySslTryToSend=-1 bSslAllSent=-1 bAllSent=0
16:32:02.516 client loSslInfo 02424090 BIO_ctrl_pending(nbio) = 33
[612]
16:32:02.516 client loSslInfo 02424090 TriggerEvent sslFdWrite 512
16:32:02.516 client loSslInfo 02424090 BIO_ctrl_pending(sslbio) =
0 [613]
16:32:02.516 client loSslInfo 02424090
BIO_ctrl_get_write_guarantee(nbio) = 4096 [614]
16:32:02.516 client loSslInfo 02424090 SslAsyncSelect 512, 2 FD_WRITE
16:32:02.516 client loSslInfo 02424090
TCustomSslWSocket.Do_FD_WRITE 512
16:32:02.516 client loSslInfo 02424090 BIO_ctrl_pending(nbio) = 33
[615]
16:32:02.516 client loSslInfo 02424090 BIO_read(nbio, 0x18BCE4,
33) = 33 [616]
16:32:02.517 client loSslInfo 02424090 my_RealSend (0x200,
1621220, 33) = 33 [617]
16:32:02.517 client loSslInfo 02424090 BIO_ctrl_pending(nbio) = 0
[618]
16:32:02.517 client loSslInfo 02424090 BIO_ctrl_pending(nbio) = 0
[619]
16:32:02.517 client loSslInfo 02424090 TriggerEvents 512 SslState:
SSL_ST_OK // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1
MaySslTryToSend=-1 bSslAllSent=-1 bAllSent=-1
16:32:02.517 client loSslInfo 02424090 BIO_ctrl_pending(nbio) = 0
[620]
16:32:02.517 client loSslInfo 02424090 BIO_ctrl_pending(sslbio) =
0 [621]
16:32:02.517 client loSslInfo 02424090
BIO_ctrl_get_write_guarantee(nbio) = 4096 [622]
... deafening silence
The source of the application, plus all required stuff from ICS, plus
the OpenSSL libraries and certificates can be downloaded here:
http://www.xs4all.nl/~bosma/SSL_Server_Client_test_source.zip
A compiled version, plus the OpenSSL libraries and certificates can be
downloaded here:
http://www.xs4all.nl/~bosma/SSL_Server_Client_test_binary.zip
The application can also be compiled without USE_SSL, then it'll work fine.
Sorry for the long post, I tried to keep it as brief as possible, I hope
someone can help or point me into the right direction :)
thanks in advance!
Merijn
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
