Re: [twsocket] using Twsocket with a few thousand clients
Hello, I already had a similar problem. If you are on Windows plateform, there a limit of message that can be queued, for so many user maybe you reached that one. That was my case with only 30 clients but a lot of small data sent for each one. When i reach this limit the TWSocketServer simply do not accept no connection and totally stop working, but the rest of my program was working perctly. To avoid that i simply avoid to send lot of message so i concatenated to call the minimum of Write(String); MSDN Source : https://msdn.microsoft.com/fr-fr/library/windows/desktop/ms644944(v=vs.85).a spx There is a limit of 10,000 posted messages per message queue. This limit should be sufficiently large. If your application exceeds the limit, it should be redesigned to avoid consuming so many system resources. To adjust this limit, modify the following registry key. Good luck ! Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de François Piette Envoyé : dimanche 12 août 2018 17:02 À : 'ICS support mailing' Objet : Re: [twsocket] using Twsocket with a few thousand clients I forget to say that to support a few thousand simultaneous client, it is likely that yu should pour TWSocketServer code into a thread to service, let's say, 500 client per thread. And you should probably NOT use the main thread to run TWSocketServer code. Be aware that ICS is asynchronous and it will multitask (cooperatively) nicely within a thread and serve a lot of connections in a single thread. But this is not infinite : Windows has his limits (Nothing in ICS limit the number of connections nor the number of threads). If you are used with some other socket component, you should probably forget everything you know. ICS has a totally and unique programming model. What is good for a synchronous blocking socket is NOT what you should do with ICS which is asynchronous, non-blocking and event driven. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Install V8.50 on C++ Builder 10.2.1
Hello Everyone,
First, thanks for all your beautiful work on this library !!!
I try today the new Tokyo 10.2.1 but I cannot install ICS v8.50 on C++
Builder 10.2 update 1.
1) There are no Install up than C++ Builder Xe3
I tried with Delphi 10.2 Tokyo : D102Install.groupproj but when I open
the project it tell me that I havent the Deplhi personality.
So I go with C++ Builder XE3 : CBXe3InstallVclFmx.groupproj
2) I build OverbyteIcsCBX3Run in release => Ok
3) I install design packages :
There is an error in OverbyteIcsWndControl line 149 and OverbyteIcsFtpCli
line 1141:
{$IFDEF RTL_NAMESPACES}Vcl.Forms{$ELSE}Forms{$ENDIF},
[Pascal Error] OverbyteIcsWndControl.pas(149, 48): F2613 Unit 'Forms' not
found.
I replaced it with
{$IFDEF RTL_NAMESPACES}Vcl.Forms{$ELSE}Vcl.Forms{$ENDIF},
But now there are lot of error with :
[ilink32 Error] Error: 'CertOpenSystemStoreW' externe non résolu, référencé
depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertFreeCertificateChainEngine' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertFreeCertificateContext' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertCloseStore' externe non résolu, référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertOpenStore' externe non résolu, référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertCreateCertificateChainEngine' externe non
résolu, référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertFindCertificateInStore' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertAddCertificateContextToStore' externe non
résolu, référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertCreateCertificateContext' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertGetCertificateChain' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CertFreeCertificateChain' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
[ilink32 Error] Error: 'CryptUIDlgViewCertificateW' externe non résolu,
référencé depuis
C:\LOGICIELSC\BUILDERXE10.2\COMMON\SCCOMPONENTS\ICS850\LIB\RELEASE\WIN32\XE3
\OVERBYTEICSMSSSLUTILS.OBJ
I tried to remove USE_SSL from compiler options but there are still the
sames errors.
Thank you for your help !
Best Regards,
Moro Alexandre.
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
Hello Everyone, I think i found a solution... First, i will to to explain what i understand. How works my applications : It connects (TWSocket) to 3 remote devices which provide data (30 chars) every 10 seconds each. It binds a server (TWSocketServer), about 8 clients are connected on it. This application compute the data from the remote device, and send the results to the 8 clients. All the flow is based on a TTimer (apart the OnDataAvaible of the server and data retreived from the client, they push data in a FIFO). The problem : After many hours (26 hours), when the clients resynchronize all the data or a new client connects to the server, the application resend all the dat to them : It is done in the TTimer event. So here, the application freeze a little time to compute and send the data (maybe 5-20 seconds) : A) If I a call the SendStr to send my data, everything is break : - Data are not sent to the clients. - I cannot etablish a new connection to my TWServerSocket (the connection is refused). - But the remote device are not disconnected. - each TWSocket says "i'm always connected". - remote device tell the same thing. - Netstat confirm. B) If i comment the SendStr, so i only compute my data, the application freeze less time (below 10 seconds). - Connection to the remote device still retrieving data. - I can always etablish new connection to my TWSocketServer. - ClientCount return by TWServerSocket is fine. The solution (or workaround !) : I simply replace TWSocketThrdServer and TWSocketThrdClient instead of TWSocketServer and TWSocketClient, and it seems to not break anything now (youhou) !!! Why ? I think that freezing the main thread breaks a lot of thing... But until here (and for about 10years), everything worked fine. For me, the "only" thing we change is that we updated from BCB6 to XE3 and to ICS v8.16 (i mean our source code stay the same). I will make a lot, lot of test to see the difference with the threaded server/client. On some application i see that they use a TAntiFreeze component maybe it can help too, but i did not try. @Tim Hyde : Thanks a lot to point me to your old thread, i was thinking that the TWSocketServer and TWSocket already implementing thread fonctionnality. And i think you have got the solution... @Angus I undestand that i'm not very clear in all the description, but i'm not English native, and i really provide you all what i have !!! I really know that it is frustrating when someone say my program do not work (and a most part of the time this is not the case...), so i understand your reponse, so i really do not want to say "ICS do not work", but i need help with your program, so please you better know the best practice to implement it ! So thanks for the works ! Another last "problem" : OverbyteIcsWSocketTS.pas is not included in the CBXe3Install nor in the CBXe3VclFmxInstall. The compiler ask for Form.dcu or Controls.dcu depending on what i'm compiling (i'm a really newbie for that...) Best regards, Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de François Piette Envoyé : samedi 3 décembre 2016 17:13 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello Alexandre, There is nothing preventing data from a TTimer event. Are you using "sync" functions or "async" functions ? Are you using wait loops ? When you send data from TTimer event, is this "transparent" to the protocol you are using ? Maybe the data send from TTimer interfere with data sent (or received) from other parts of your program? -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : vendredi 2 décembre 2016 13:44 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello ! I finaly found where comes from the problem : When the clients connects to my server, i send to everyone a lot of data from a TTimer. If i remove the SendStr() call, everything works fine !!! So please, 1) Can you explain me what is wrong in calling SendStr from a TTimer ? 2) How can this call, block every differents object based on TWSocket, server object and client object ? 3) Why there are no error event nor exception throw... Solution : 1) I tried to set mpWSocketServer->MultiThread = true; But it doesn't change anything... 2) Do i have to implement a Thread for all the SendStr be apart from the main thread ? I create a Thread, pass the TWSocketServer object as parameters, and a FIFO list with event. When i complete this list, the thread process the new data and SendStr... I'm shocked that this call can interfere seperate object... Thank you ! Best regards, Moro Ale
Re: [twsocket] TWSocket Client Timeout.
Hello Everyone, Again, i'm back with this problem with some news... I have reproduce the problem : The configuration : 1 process based on ICS 8.34 with : a socket server with 4 clients connected (they only receive data) (TCP). 3 connection etablished to 3 different socket server from network devices (TCP). Everything worked for 28 Hours. I started another application which freezed the computer for 1-2 seconds, and now the problem happen : Problems : 1) No data output the application No more data is sent from my application to the 4 clients. (but everyone says it is well connected). 2) No more data are computed by my application. I do not received new data. (everyone is connected (netstat), and wireshark show me the data are well received by the computer). What can i see : The processus is not freezed. All clients show that all TCP connection from and to everyone are in etablish state (from netstat). Wireshark show that data from the 3 clients to the processus are well received on the network interface. TWSocket client object say that i'm still connected to the 3 clients ! I reproduce this bug for the first time since more than one year and outside of any IDE so i cannot search were does it really stuck... I will now try to reproduce the problem from an IDE... In this time, did you already ear something like that with ICS ? Have you any idea where data can be blocked, and in this state what can i do more than inspecting by wireshark and netstat ??? Thank you. Best regards, Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : mardi 11 octobre 2016 08:37 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello everyone, I'm back, and i got good news, i finaly found what was going on : It is simply a problem half open connection : The connection between the server and the client (using TWSocket) is forwarded althrough a Wifi connection (based on 868Mhz, which is really unstable). Sometimes when the server sent somes frames to the client, the server doesn't receive the ACK and the server close the connection from the client. But, this client doesn't know it, so TWSocket thinks the connection is still open... I checked it with netstat on both server and client and it is very clear now ! To better handle this situation, i changed KeepAlive timeout like that : mpWSocket->KeepAliveOnOff = wsKeepAliveOnCustom; KeepAliveTime = 25 * mpWSocket->1000; And now, we clearly see that the socket closed itself after these 25 seconds without data received (when the Wifi is down) and try to reconnect... Sorry to loose your time, maybe it can help someone encoutered the same problem. Best, Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : mercredi 5 octobre 2016 08:44 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello Stephen, Thank you for sharing and giving me a way to search about. I searched about theses function in my whole source code but these are not used. I looked for something similar like low level code but i did not really find something interessting... So wait and see... -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Stephen Dickason Envoyé : mardi 4 octobre 2016 18:52 À : ICS support mailing Objet : Re: [twsocket] TWSocket Client Timeout. Please pardon me for butting in. I can share what I found broke ICS for me and you can check if you have anything similar. I used a separate piece of low level code (not ICS) in my about form that displayed the host name and IP address of the user's PC, it used winsock.gethostname, WSAStartup($101,...) and WSACleanup. (I don't remember which one was the culprit.) This I found broke any open socket connections that I had and only way to see if they were broken was something like cmd's netstat -a. It was rather a bother, as not many people check the about box, so was quite random, showed no errors and was pretty annoying. I fixed it a long time ago by using internal ICS functions instead. Hope it helps. Stephen Dickason Senior Developer - Managed Services Email: sdicka...@elcb.co.za ELCB Information Services (Pty) Ltd Customer Service Email e...@elcb.co.za * www.elcb.co.za E A S T L O N D O N Tel: +27(43) 704 0700 Fax: +27(43) 704 0701 J O H A N N E S B U R G Tel: +27(11) 879 6179 Fax: +27(11) 454 0384 P O R T E L I Z A B E T H Tel: +27(41) 373 0529 Fax: +27(86) 650 0135 Disclaimer > -Original Message- > From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of > Moro Alexandre > Sent: Tuesday, October 4, 2016 17:19 > To: 'ICS support mailing' > Subject: Re: [twsocket] TWSocket Client Timeout. > > Hi Angus, > > Sorry to bother you, please find more details : > > In fact, an user use our application
Re: [twsocket] TWSocket Client Timeout.
I really know these are very poor details, but like i say, this is a report from a customer and not a developper, sadly, i did not reproduce this problem more than one year after the first time it happens... The new thing i can say is that the user must restart the program to retrieve the data again. Since our source code logic do not change between the update of ICS it let me think that it is related to that. I asked you to see if, maybe, you know a way to search about, like Stephen Dickason and his WSAStartup... Apparently, this fix you provide can really be linked to my problem. When i say the connection is still open, i mean : a green light on the form based on IsConnected() are still green (Informations comes from user not developpers)! So i already upgraded to the latest version and i going to try it on the user computer to try to reproduce the problem myself... But i believe in this fix ! Thanks for your time. Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Angus Robertson - Magenta Systems Ltd Envoyé : mardi 4 octobre 2016 17:42 À : twsocket@lists.elists.org Objet : Re: [twsocket] TWSocket Client Timeout. > Sorry to bother you, please find more details : Sorry, still no real detail of any use for diagnostics. I suggest you upgrade to the latest ICS V8.34, we don't use SVN commit versions, and add more diagnostics into your application to try and trace your problem. The timeout error I fixed related to a idle connection being deliberately closed by ICS when it was not actually idle (because a counter was not updated) but you say the connection is not closed. I have TCP connections running continually for days or weeks without any data being lost, it is unlikely to be an ICS problem. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
Hello Angus, Thanks for your fast response. But no other application could effect this property. Fo sure, I understand that no other application can change this var in my application (normaly), but i'm talking at i higher level... However other applications can change the TCP/IP timer parameters This is what i'm thinking about. But since my TWSocket client only receive data and never sent any data, i believe an antivirus or a firewall or windows with changed TCP/IP parameters, can thing it is a bad connection and close it after a while (5hours in my case). A normal TCP/IP socket never tries to reconnect, your applications controls all that. You are not talking about TWSocket... Because i doesn't implement anything in this way in my program and TWSocket already manage it correclty : If I etablish a connection to a server then stop and restart the server TWSocket client retablish the connection automatically... Anyway, i will try to remove all applications who can act on ethernet connection first... Thanks. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Angus Robertson - Magenta Systems Ltd Envoyé : jeudi 11 juin 2015 11:02 À : twsocket@lists.elists.org Objet : Re: [twsocket] TWSocket Client Timeout. Is it possible that the default 0 timeout on TWSocket will be replace by another program like Anti Virus or Firewall ? I see that by default TimeoutIdle is set to zero, does it can change something if a force this to the maximum unsigned int value ? The TimeoutIdle property is part of an undocumented, experimental TCustomTimeoutWSocket component. Without examining the code, I've no idea what this property is designed to do, never used it. But no other application could effect this property. Generally, Windows TCP/IP registry setting define to connection timeout, there is no inherent idle timeout in TCP/IP. However other applications can change the TCP/IP timer parameters, generally not a good idea. And other applications might close what they consider to be an idle connection. Another way : I there a maximum number of reconnection allowed during a connection ? A normal TCP/IP socket never tries to reconnect, your applications controls all that. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] TWSocket Client Timeout.
Hello, Is it possible that the default 0 timeout on TWSocket will be replace by another program like Anti Virus or Firewall ? My case : I got 4 differents clients socket connected to 4 differents server on a first computer. After about 5 hours, someone lost the connection (between 3 and 4 clients). On a second computer with the same setup, connections are still working. I see that by default TimeoutIdle is set to zero, does it can change something if a force this to the maximum unsigned int value ? Another way : I there a maximum number of reconnection allowed during a connection ? Thanks you. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Server crash from client.
Hello Angus,
Thanks for your reply.
These are the number you provide in your changelog :
r1250 | angus | 2015-03-26 12:07:09 + (Thu, 26 Mar 2015) | 5 lines
Mar 26, 2015 V8.16
r1196 | aguser | 2013-12-01 15:06:38 + (Sun, 01 Dec 2013) | 1 line
I do not handle any client since this management is internal to
TWSocketServer.
The really only thing i do is to send String with SendStr :
mpWSocketServer-Addr = 0.0.0.0;
mpWSocketServer-Port = mHostPort;
mpWSocketServer-Proto = tcp;
mpWSocketServer-Listen();
And when i send data :
for (int i = 0; i mpWSocketServer-ClientCount; i++)
{
mpWSocketServer-Client[i]-SendStr(frame);
mpWSocketServer-Client[i]-Flush();
}
I added the OnBGexception on both TWSocketServer and client :
void __fastcall TServerSocketEngine::ClientBgException(TObject *Sender,
Exception *E,
bool CanClose)
{
CanClose = TRUE; // Goodbye client !
}
In the example i can read :
// This event handler is called when a client socket experience a background
// exception. It is likely to occurs when client aborted connection and data
// has not been sent yet.
Now, i first receive the Stack overflow Exception,
then the ClientBgException() is called with this Overflow,
and at least, a couple of seconds later, the program crash on an EAccess
Violation and can't be recovered.
C++ XE3 focus me on the line 16233 of the file OverbyteIcsWSocket.pas below
the line of begin :
procedure TCustomSslWSocket.Do_FD_WRITE(var Msg: TMessage);
var
Len: Integer;// How much to send
Buffer : array [0..16383] of AnsiChar;
NumRead: Integer;
NumSent: Integer;
Err: Longword;
begin
if (not FSslEnable) or (FSocksState socksData) or
(FHttpTunnelState htsData) then begin
inherited Do_FD_WRITE(msg);
Exit;
end;
I really don't know what can i do anymore !
PS : Is it normal that i can't compile ICS in release mode ?
Thanks.
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
[twsocket] Server crash from client.
Hello developpers,
My program crash with a Stack Overflow exception wich i can't catch but i
can easely reproduce :
I start a server program S and a client program C the client is started in
debug mode.
Both S and C use TWSocket class as server and client.
Server :
Send small chunk of text data with :
mpWSocket-SendStr(frame);
mpWSocket-Flush();
The client retrieve data with :
mpWSocket-OnDataAvailable = DataAvailable;
So the program works very well,
but if the client program C blocked on a breakpoint, after a while (maybe
500 chunks of 80 characters sent by the server S)
the server S crash, i can see a stack overflow error by starting S in debug
mode and in release mode window show an alert The program has stopped
working .
The problem is that, apparently, i can crash the server from the client,
here with a breakpoint but if i a popup a Messagebox it block the client
program and crash the server S in the same way.
Isn't it a big security problem ?
Maybe it is just property to set on the server to catch this exception but
wich one ?
PS 1 : every write are enclosed with a try { } catch(...) {}
PS 2 : It happens with OverbyteICS r1196 and r1250 in debug mode (I can't
compile and install release mode because it doesn't find controls.dcu on
both versions).
Thanks in advance for your help.
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
