Re: [twsocket] TWSocket Client Timeout.
Hello Everyone, I think i found a solution... First, i will to to explain what i understand. How works my applications : It connects (TWSocket) to 3 remote devices which provide data (30 chars) every 10 seconds each. It binds a server (TWSocketServer), about 8 clients are connected on it. This application compute the data from the remote device, and send the results to the 8 clients. All the flow is based on a TTimer (apart the OnDataAvaible of the server and data retreived from the client, they push data in a FIFO). The problem : After many hours (26 hours), when the clients resynchronize all the data or a new client connects to the server, the application resend all the dat to them : It is done in the TTimer event. So here, the application freeze a little time to compute and send the data (maybe 5-20 seconds) : A) If I a call the SendStr to send my data, everything is break : - Data are not sent to the clients. - I cannot etablish a new connection to my TWServerSocket (the connection is refused). - But the remote device are not disconnected. - each TWSocket says "i'm always connected". - remote device tell the same thing. - Netstat confirm. B) If i comment the SendStr, so i only compute my data, the application freeze less time (below 10 seconds). - Connection to the remote device still retrieving data. - I can always etablish new connection to my TWSocketServer. - ClientCount return by TWServerSocket is fine. The solution (or workaround !) : I simply replace TWSocketThrdServer and TWSocketThrdClient instead of TWSocketServer and TWSocketClient, and it seems to not break anything now (youhou) !!! Why ? I think that freezing the main thread breaks a lot of thing... But until here (and for about 10years), everything worked fine. For me, the "only" thing we change is that we updated from BCB6 to XE3 and to ICS v8.16 (i mean our source code stay the same). I will make a lot, lot of test to see the difference with the threaded server/client. On some application i see that they use a TAntiFreeze component maybe it can help too, but i did not try. @Tim Hyde : Thanks a lot to point me to your old thread, i was thinking that the TWSocketServer and TWSocket already implementing thread fonctionnality. And i think you have got the solution... @Angus I undestand that i'm not very clear in all the description, but i'm not English native, and i really provide you all what i have !!! I really know that it is frustrating when someone say my program do not work (and a most part of the time this is not the case...), so i understand your reponse, so i really do not want to say "ICS do not work", but i need help with your program, so please you better know the best practice to implement it ! So thanks for the works ! Another last "problem" : OverbyteIcsWSocketTS.pas is not included in the CBXe3Install nor in the CBXe3VclFmxInstall. The compiler ask for Form.dcu or Controls.dcu depending on what i'm compiling (i'm a really newbie for that...) Best regards, Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de François Piette Envoyé : samedi 3 décembre 2016 17:13 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello Alexandre, There is nothing preventing data from a TTimer event. Are you using "sync" functions or "async" functions ? Are you using wait loops ? When you send data from TTimer event, is this "transparent" to the protocol you are using ? Maybe the data send from TTimer interfere with data sent (or received) from other parts of your program? -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : vendredi 2 décembre 2016 13:44 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello ! I finaly found where comes from the problem : When the clients connects to my server, i send to everyone a lot of data from a TTimer. If i remove the SendStr() call, everything works fine !!! So please, 1) Can you explain me what is wrong in calling SendStr from a TTimer ? 2) How can this call, block every differents object based on TWSocket, server object and client object ? 3) Why there are no error event nor exception throw... Solution : 1) I tried to set mpWSocketServer->MultiThread = true; But it doesn't change anything... 2) Do i have to implement a Thread for all the SendStr be apart from the main thread ? I create a Thread, pass the TWSocketServer object as parameters, and a FIFO list with event. When i complete this list, the thread process the new data and SendStr... I'm shocked that this call can interfere seperate object... Thank you ! Best regards, Moro Ale
Re: [twsocket] TWSocket Client Timeout.
Hello Alexandre, There is nothing preventing data from a TTimer event. Are you using "sync" functions or "async" functions ? Are you using wait loops ? When you send data from TTimer event, is this "transparent" to the protocol you are using ? Maybe the data send from TTimer interfere with data sent (or received) from other parts of your program? -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : vendredi 2 décembre 2016 13:44 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello ! I finaly found where comes from the problem : When the clients connects to my server, i send to everyone a lot of data from a TTimer. If i remove the SendStr() call, everything works fine !!! So please, 1) Can you explain me what is wrong in calling SendStr from a TTimer ? 2) How can this call, block every differents object based on TWSocket, server object and client object ? 3) Why there are no error event nor exception throw... Solution : 1) I tried to set mpWSocketServer->MultiThread = true; But it doesn't change anything... 2) Do i have to implement a Thread for all the SendStr be apart from the main thread ? I create a Thread, pass the TWSocketServer object as parameters, and a FIFO list with event. When i complete this list, the thread process the new data and SendStr... I'm shocked that this call can interfere seperate object... Thank you ! Best regards, Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : vendredi 2 décembre 2016 08:14 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. I trying to investigate more and more and i see that i cannot open new sockets on the server, it refuse the connection, below is the packets captured in wireshark : Document joint : spuriousretransmission.pcapng The application i openned who crashed all the twsocket open two new socket on this server ! And in netstat the server socket are still in listenning state, (it happen on localhost and from a remote computer). Moreover the last 4 connection are in FIN_WAIT_2 Maybe it is a start... Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Angus Robertson - Magenta Systems Ltd Envoyé : jeudi 1 décembre 2016 16:54 À : twsocket@lists.elists.org Objet : Re: [twsocket] TWSocket Client Timeout. > In this time, did you already ear something like that with ICS ? > Have you any idea where data can be blocked, and in this state what > can i do more than inspecting by wireshark and netstat ??? If I understand you correctly, you want to keep remote TCP/IP sockets connected for several days at a time, presumably over a LAN or maybe a WAN? There are so many ways a connection can be lost, routers and switches, hardware issues, software interference, Windows Updates, etc. Even with keep alive, TCP/IP sockets can say connected but not be, the only reliable way is to send and echo data, a proper handshake. The probability of it being a bug in ICS that no-one else has noticed is highly unlikely, but not impossible, as explained previously I did fix a timeout issue that caused a socket to be closed earlier this year, which I found in one of my applications. You really need to find a way of bullet proofing your application, so it reconnects if nothing is received for x second/minutes/hours, or use regular handshaking for a more reliable and shorter timeout. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
> If i remove the SendStr() call, everything works fine !!! So what did you replace it with? There is nothing wrong with sending data, applications do it all the time. Sorry, you are not really providing adequate information about your application and which components or even OS, you are using to make any real suggestions. It's very unlikely threads will be necessary, unless you are blocking ICS somewhere. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
> In this time, did you already ear something like that with ICS ? > Have you any idea where data can be blocked, and in this state > what can i do more than inspecting by wireshark and netstat ??? If I understand you correctly, you want to keep remote TCP/IP sockets connected for several days at a time, presumably over a LAN or maybe a WAN? There are so many ways a connection can be lost, routers and switches, hardware issues, software interference, Windows Updates, etc. Even with keep alive, TCP/IP sockets can say connected but not be, the only reliable way is to send and echo data, a proper handshake. The probability of it being a bug in ICS that no-one else has noticed is highly unlikely, but not impossible, as explained previously I did fix a timeout issue that caused a socket to be closed earlier this year, which I found in one of my applications. You really need to find a way of bullet proofing your application, so it reconnects if nothing is received for x second/minutes/hours, or use regular handshaking for a more reliable and shorter timeout. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
Hello Everyone, Again, i'm back with this problem with some news... I have reproduce the problem : The configuration : 1 process based on ICS 8.34 with : a socket server with 4 clients connected (they only receive data) (TCP). 3 connection etablished to 3 different socket server from network devices (TCP). Everything worked for 28 Hours. I started another application which freezed the computer for 1-2 seconds, and now the problem happen : Problems : 1) No data output the application No more data is sent from my application to the 4 clients. (but everyone says it is well connected). 2) No more data are computed by my application. I do not received new data. (everyone is connected (netstat), and wireshark show me the data are well received by the computer). What can i see : The processus is not freezed. All clients show that all TCP connection from and to everyone are in etablish state (from netstat). Wireshark show that data from the 3 clients to the processus are well received on the network interface. TWSocket client object say that i'm still connected to the 3 clients ! I reproduce this bug for the first time since more than one year and outside of any IDE so i cannot search were does it really stuck... I will now try to reproduce the problem from an IDE... In this time, did you already ear something like that with ICS ? Have you any idea where data can be blocked, and in this state what can i do more than inspecting by wireshark and netstat ??? Thank you. Best regards, Moro Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : mardi 11 octobre 2016 08:37 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello everyone, I'm back, and i got good news, i finaly found what was going on : It is simply a problem half open connection : The connection between the server and the client (using TWSocket) is forwarded althrough a Wifi connection (based on 868Mhz, which is really unstable). Sometimes when the server sent somes frames to the client, the server doesn't receive the ACK and the server close the connection from the client. But, this client doesn't know it, so TWSocket thinks the connection is still open... I checked it with netstat on both server and client and it is very clear now ! To better handle this situation, i changed KeepAlive timeout like that : mpWSocket->KeepAliveOnOff = wsKeepAliveOnCustom; KeepAliveTime = 25 * mpWSocket->1000; And now, we clearly see that the socket closed itself after these 25 seconds without data received (when the Wifi is down) and try to reconnect... Sorry to loose your time, maybe it can help someone encoutered the same problem. Best, Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Moro Alexandre Envoyé : mercredi 5 octobre 2016 08:44 À : 'ICS support mailing' Objet : Re: [twsocket] TWSocket Client Timeout. Hello Stephen, Thank you for sharing and giving me a way to search about. I searched about theses function in my whole source code but these are not used. I looked for something similar like low level code but i did not really find something interessting... So wait and see... -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Stephen Dickason Envoyé : mardi 4 octobre 2016 18:52 À : ICS support mailing Objet : Re: [twsocket] TWSocket Client Timeout. Please pardon me for butting in. I can share what I found broke ICS for me and you can check if you have anything similar. I used a separate piece of low level code (not ICS) in my about form that displayed the host name and IP address of the user's PC, it used winsock.gethostname, WSAStartup($101,...) and WSACleanup. (I don't remember which one was the culprit.) This I found broke any open socket connections that I had and only way to see if they were broken was something like cmd's netstat -a. It was rather a bother, as not many people check the about box, so was quite random, showed no errors and was pretty annoying. I fixed it a long time ago by using internal ICS functions instead. Hope it helps. Stephen Dickason Senior Developer - Managed Services Email: sdicka...@elcb.co.za ELCB Information Services (Pty) Ltd Customer Service Email e...@elcb.co.za * www.elcb.co.za E A S T L O N D O N Tel: +27(43) 704 0700 Fax: +27(43) 704 0701 J O H A N N E S B U R G Tel: +27(11) 879 6179 Fax: +27(11) 454 0384 P O R T E L I Z A B E T H Tel: +27(41) 373 0529 Fax: +27(86) 650 0135 Disclaimer > -Original Message- > From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of > Moro Alexandre > Sent: Tuesday, October 4, 2016 17:19 > To: 'ICS support mailing' > Subject: Re: [twsocket] TWSocket Client Timeout. > > Hi Angus, > > Sorry to bother you, please find more details : > > In fact, an user use our application
Re: [twsocket] TWSocket Client Timeout.
I really know these are very poor details, but like i say, this is a report from a customer and not a developper, sadly, i did not reproduce this problem more than one year after the first time it happens... The new thing i can say is that the user must restart the program to retrieve the data again. Since our source code logic do not change between the update of ICS it let me think that it is related to that. I asked you to see if, maybe, you know a way to search about, like Stephen Dickason and his WSAStartup... Apparently, this fix you provide can really be linked to my problem. When i say the connection is still open, i mean : a green light on the form based on IsConnected() are still green (Informations comes from user not developpers)! So i already upgraded to the latest version and i going to try it on the user computer to try to reproduce the problem myself... But i believe in this fix ! Thanks for your time. Alexandre. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Angus Robertson - Magenta Systems Ltd Envoyé : mardi 4 octobre 2016 17:42 À : twsocket@lists.elists.org Objet : Re: [twsocket] TWSocket Client Timeout. > Sorry to bother you, please find more details : Sorry, still no real detail of any use for diagnostics. I suggest you upgrade to the latest ICS V8.34, we don't use SVN commit versions, and add more diagnostics into your application to try and trace your problem. The timeout error I fixed related to a idle connection being deliberately closed by ICS when it was not actually idle (because a counter was not updated) but you say the connection is not closed. I have TCP connections running continually for days or weeks without any data being lost, it is unlikely to be an ICS problem. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
Please pardon me for butting in. I can share what I found broke ICS for me and you can check if you have anything similar. I used a separate piece of low level code (not ICS) in my about form that displayed the host name and IP address of the user's PC, it used winsock.gethostname, WSAStartup($101,...) and WSACleanup. (I don't remember which one was the culprit.) This I found broke any open socket connections that I had and only way to see if they were broken was something like cmd's netstat -a. It was rather a bother, as not many people check the about box, so was quite random, showed no errors and was pretty annoying. I fixed it a long time ago by using internal ICS functions instead. Hope it helps. Stephen Dickason Senior Developer - Managed Services Email: sdicka...@elcb.co.za ELCB Information Services (Pty) Ltd Customer Service Email e...@elcb.co.za * www.elcb.co.za E A S T L O N D O N Tel: +27(43) 704 0700 Fax: +27(43) 704 0701 J O H A N N E S B U R G Tel: +27(11) 879 6179 Fax: +27(11) 454 0384 P O R T E L I Z A B E T H Tel: +27(41) 373 0529 Fax: +27(86) 650 0135 Disclaimer > -Original Message- > From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of Moro > Alexandre > Sent: Tuesday, October 4, 2016 17:19 > To: 'ICS support mailing' > Subject: Re: [twsocket] TWSocket Client Timeout. > > Hi Angus, > > Sorry to bother you, please find more details : > > In fact, an user use our application to retrieve data from a device by a TCP > connection throught TWSocket. > > The problem is that after a certain time (between 5 and 8 hours) the data stop > being received but TWSocket still indicate that it is connected. > > The user has reproduce the problem multiple times. > > This problem only appears since one year : when we have updated C++ Builder > and ICS : > > From Borland Builder 6 to Embarcadero XE3 From ICS 5.25 to ICS r1196. > > I do not have more details since i can not reproduce this problem myself. > But we are going to reproduce it this weekend. > > Finaly, i just see that you have fix some problems with timeout and > WSAESHUTDOWN in the version r1280 and r1279, so since it is related to time > maybe it fix my problem of 5/8 Hours... > > I cannot reproduce the original problem, unfortunately i cannot try it for the > moment ! > > Thanks again !!! > > Alexandre. > > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocket Client Timeout.
Hello Angus, Thanks for your fast response. But no other application could effect this property. Fo sure, I understand that no other application can change this var in my application (normaly), but i'm talking at i higher level... However other applications can change the TCP/IP timer parameters This is what i'm thinking about. But since my TWSocket client only receive data and never sent any data, i believe an antivirus or a firewall or windows with changed TCP/IP parameters, can thing it is a bad connection and close it after a while (5hours in my case). A normal TCP/IP socket never tries to reconnect, your applications controls all that. You are not talking about TWSocket... Because i doesn't implement anything in this way in my program and TWSocket already manage it correclty : If I etablish a connection to a server then stop and restart the server TWSocket client retablish the connection automatically... Anyway, i will try to remove all applications who can act on ethernet connection first... Thanks. -Message d'origine- De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de Angus Robertson - Magenta Systems Ltd Envoyé : jeudi 11 juin 2015 11:02 À : twsocket@lists.elists.org Objet : Re: [twsocket] TWSocket Client Timeout. Is it possible that the default 0 timeout on TWSocket will be replace by another program like Anti Virus or Firewall ? I see that by default TimeoutIdle is set to zero, does it can change something if a force this to the maximum unsigned int value ? The TimeoutIdle property is part of an undocumented, experimental TCustomTimeoutWSocket component. Without examining the code, I've no idea what this property is designed to do, never used it. But no other application could effect this property. Generally, Windows TCP/IP registry setting define to connection timeout, there is no inherent idle timeout in TCP/IP. However other applications can change the TCP/IP timer parameters, generally not a good idea. And other applications might close what they consider to be an idle connection. Another way : I there a maximum number of reconnection allowed during a connection ? A normal TCP/IP socket never tries to reconnect, your applications controls all that. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] TWSocket Client Timeout.
Hello, Is it possible that the default 0 timeout on TWSocket will be replace by another program like Anti Virus or Firewall ? My case : I got 4 differents clients socket connected to 4 differents server on a first computer. After about 5 hours, someone lost the connection (between 3 and 4 clients). On a second computer with the same setup, connections are still working. I see that by default TimeoutIdle is set to zero, does it can change something if a force this to the maximum unsigned int value ? Another way : I there a maximum number of reconnection allowed during a connection ? Thanks you. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
