Re: [twsocket] Sending data from multiple threads on SSL doesn't work
> It happens if I'm sending from a thread or even only from the > main thread (from a timer), please check this demo app: I look very briefly at your demo, it uses the non-thread aware socket server, and creates one single thread to send data to all clients, with no locking attempts or even a message loop in the thread. Your application should be based on TSslWSocketThrdServer which creates one thread per client, and handles all the threaded messages correctly. You should not need to create any threads yourself, although I guess extra threads within a client would not harm provided they do not use ICS functions. As I said before, I've never used TSslWSocketThrdServer myself, and it does not yet support multi-listen or advanced SSL stuff like IcsHosts which simplify SSL programming considerable. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Sending data from multiple threads on SSL doesn't work
> I didn't change any ICS code, but only my derived class... do you > think that by always adding to the send buffer PutDataInSendBuffer) > and posting FD_WRITE instead of calling TryToSend could cause any > issues? There are often several ways of doing something, if extensive testing does not show any problems, then it works. But you have still not answered my question about SSL version or the locking unit. ICS V8.53 came with two versions of OpenSSL. It would be worth testing both 1.0.2 and 1.1.0 to see if the behaviour is different, since threading support changed completely between them. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Sending data from multiple threads on SSL doesn't work
I understand
But non-threaded is not an option.. our application is a high performance
video management system, it must be multi-threaded, in fact we have a very
highly threaded architecture, and it would create huge bottlenecks making
it single threaded (for network message processing). This specific part of
our application, is the part that sends video from server to clients. I
assign the socket to a message processing thread that manages sending and
receiving data for this section. I don't necessarily have to send data from
different threads, data can be sent from a single thread (but not main
thread) that is assigned to the socket.
But.. I found a solution to fix my initial problem (sending data from two
different threads).
Can you please validate if this would cause any issues?
I have a custom client class (for TWSocketServer) that is derived
from TWSocketClient that is instantiated by the server for every client
socket that connects to it.
I overwrote the Send function with the following code that checks for Ssl,
if Ssl is disabled then it would call inherited.. but it it is enabled,
instead of calling TryToSend directly from the calling thread (that is
causing the issue), it would only add the data to the send buffer and then
post FD_WRITE message, and the thread assigned to the client socket would
pick it up later and send the buffered data from its context.
I didn't change any ICS code, but only my derived class... do you think
that by always adding to the send buffer (PutDataInSendBuffer) and posting
FD_WRITE instead of calling TryToSend could cause any issues? Otherwise,
this fixes the main issue of socket stopping writting data... I will still
investigate on disconnections...
// When working with SSL, we can't send data from different threads,
// apparently OpenSSL does not like it. It could be an issue with ICS as
well.
// To overcome the problem, when sending data with SSL enabled, instead of
// directly sending the data from the calling thread, we just put the data
// to the send buffer and post a message so it will be sent later from the
// thread that is processing the socket messages
if (not SslEnable) or
((FState <> wsConnected) and (FState <> wsSocksConnected)) then
begin
inherited;
Exit;
end;
// Add to send buffer
if Len <= 0 then
Exit(0)
else
begin
Result := Len;
PutDataInSendBuffer(Data, Len);
end;
// Post message for sending data
PostMessage(Handle,
FMsg_WM_ASYNCSELECT,
WParam(FHSocket),{ V8.08 }
IcsMakeLong(FD_WRITE, 0));
Thanks
Eric
On Wed, Oct 3, 2018 at 3:10 PM Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:
> > It happens if I'm sending from a thread or even only from the
> > main thread (from a timer), please check this demo app:
>
> Sorry, won't be looking for more bugs in ICS until the V8.57 release is
> done, it's weeks later already.
>
> I can only suggest trying to avoid threads, I don't use them and don't
> have any problems in my commercial applications.
>
> Angus
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Sending data from multiple threads on SSL doesn't work
Angus
Funny thing, when the error happens, getlasterror from SSL returns:
error:1408F10B:SSL routines:ssl3_get_record:wrong version number
The disconnection is easily reproduceable by spawning a few threads
Eric
On Wed, Oct 3, 2018 at 2:01 PM Éric Fleming Bonilha
wrote:
> Angus
>
> I'm trying to make a demo application to test this behavior, but now I'm
> stuck with another problem.. random disconnects..
>
> I'm using latest ICS 8.53 with OpenSSL that is shipped with the ICS
>
> Whenever I start sending data from server -> client, the client will
> disconnect randomly.
>
> It happens if I'm sending from a thread or even only from the main thread
> (from a timer), please check this demo app:
>
> http://www.digifort.com/temp/SSLMultithread.rar
>
> I've found that the client disconnection is happening here:
> TCustomSslWSocket.Do_FD_READ
>
> Res := my_BIO_read(FSslBio, @Dummy, 0); //Pointer(1)
> if Res < 0 then begin
> if not my_BIO_should_retry(FSslBio) then begin
>
>
> my_BIO_read returns -1 and then my_BIO_should_retry returns FALSE
>
> I see this comment on the code:
> { Get number of bytes we can receive and store in the network
> input bio. }
> { New call to BIO_ctrl_get_read_request in order to read only the
> amount }
> { of data from the socket that is needed to satisfy a read
> request, if}
> { any. Without that call I had random errors on bi-directional
> shutdowns. }
>
>
> Do you have any ideas why the client is being disconnected on this demo
> app?
>
> This demo app is very simple.. I simply send data to the connected client,
> and when data is received I increment a number of received bytes per second
>
> Eric
>
>
> On Wed, Oct 3, 2018 at 12:38 PM Angus Robertson - Magenta Systems Ltd <
> an...@magsys.co.uk> wrote:
>
>> > I'm currently using OpenSSL 1.0.2h and I tried with latest ICS I
>> > downloaded yesterday (8.53)
>>
>> Does your application include OverbyteIcsSslThrdLock.pas? That is
>> needed to make locking work with 1.0.2.
>>
>> If you use 1.1.0 or 1.1.1, that unit is not needed.
>>
>> But I don't know much about OverbyteIcsSslThrdLock, never used it
>> myself. It is used by OverbyteIcsWSocketTS.pas which I assume you are
>> using, but I've never looked at that unit, don't know if the OpenSSL
>> stuff is up to date. There is only one demo that uses it, a simple
>> socket server, not HTTP or anything easy to test.
>>
>> I have been meaning to look at a threaded web server, but never had the
>> time.
>>
>> Angus
>>
>> --
>> To unsubscribe or change your settings for TWSocket mailing list
>> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
>> Visit our website at http://www.overbyte.be
>>
>
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] Sending data from multiple threads on SSL doesn't work
Hi Angus Thanks for the reply. This is indeed a strange problem, because there's no deadlock or locking problem, and the problem does not happen in a deterministic way.. its kind of random, so I have to keep it running for a few seconds or minutes, then it will stop, maybe when a certain condition / workflow is met I'm currently using OpenSSL 1.0.2h and I tried with latest ICS I downloaded yesterday (8.53) Maybe I should update my OpenSSL library for testing? I will update it, if it still happen I will try to make a demo program that can simulate this issue.. This is beyond my pay grade.. I don't have any experience with OpenSSL Thanks Eric On Wed, Oct 3, 2018 at 3:25 AM Angus Robertson - Magenta Systems Ltd < an...@magsys.co.uk> wrote: > > I understand that ICS is thread safe when sending data because > > adding and removing data to send buffers are locked with critical > > sections.. but maybe OpenSSL does not like when we send data > > from two different threads? > > OpenSSL is thread safe, or at least has been in the past, I've tested > an FTP application sending hundreds of SSL sessions each using a > separate thread. > > But OpenSSL has changed a lot in the last two years, there is a ICS > unit OverbyteIcsSslThrdLock which used to provide OpenSSL with thread > locking for 1.0.2 and earlier, but is no longer used for 1.1.0 and > later since OpenSSL does locking internally. > > I've not tested heavy thread use for a while, since it is rarely > necessary with ICS, it just complicates applications and introduces > problems. > > Which OpenSSL and ICS versions are you using? > > Angus > > > > > -- > To unsubscribe or change your settings for TWSocket mailing list > please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket > Visit our website at http://www.overbyte.be > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Sending data from multiple threads on SSL doesn't work
> I understand that ICS is thread safe when sending data because > adding and removing data to send buffers are locked with critical > sections.. but maybe OpenSSL does not like when we send data > from two different threads? OpenSSL is thread safe, or at least has been in the past, I've tested an FTP application sending hundreds of SSL sessions each using a separate thread. But OpenSSL has changed a lot in the last two years, there is a ICS unit OverbyteIcsSslThrdLock which used to provide OpenSSL with thread locking for 1.0.2 and earlier, but is no longer used for 1.1.0 and later since OpenSSL does locking internally. I've not tested heavy thread use for a while, since it is rarely necessary with ICS, it just complicates applications and introduces problems. Which OpenSSL and ICS versions are you using? Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
