Re: [twsocket] twsocketserver linemode question
Tony Caduto wrote: I see there is a LineLimitExceeded event, but not sure how to use it. By default the component clears the internal line buffer if the end of line marker wasn't found. I would assume some attack or a suspicious client when that happens at the server side and close the connection as it is shown in the OverbyeIcsTcpSrv demo: procedure TTcpSrvForm.ClientLineLimitExceeded( Sender: TObject; Cnt : LongInt; var ClearData : Boolean); begin with Sender as TTcpSrvClient do begin Display('Line limit exceeded from ' + GetPeerAddr + '. Closing.'); ClearData := TRUE; Close; end; end; -- Arno -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
Can someone please guide me to a correct demo/sample of how to send receive files? Do you have any preference about the protocol to use ? What are your requirement ? Send and receive file is a little bit short to describe your needs. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
I hope this was clear. Not enough. see below. I need to send single file from the server and receive single file from the client (1 file at the time such as: .txt, .exe). What about file metadata such as filename and timestamp ? What about permissions to access the file at sender side and to write the file at receiver side ? I will be sending and receiving only single files and the size will be maximum of 100MB Doesn't matter. The only special case wiuld be very small files. Is it possible to do it over the socket with for example file stream? if not what are your suggestions? FTP, HTTTP, SMTP/POP3, NNTP are all capable of transmit file and they are standard protocols based on TCP sockets. You may also design your own protocol based on your own requirements which are... Basically, using HTTP client and server components is the easiest way of transfering files without the hassle of designing your own protocol. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
Thanks Francois, I probably need to get into this abit more. I have some more details.. What about file metadata such as filename and timestamp ? What about permissions to access the file at sender side and to write the file at receiver side ? -- I actualy think I could handle it on the both sides very easyly because, Upload from client to the server (log files only which are .txt) with an automated process such as client will send the log file when it is asked by the server like, server sends command: getlogfile and client answers by sending the log file into a prefixed directory placed in the server with a prefixed name. Server sends an update file which is prefixed name such as setup.exe and client will save it into a prefixed directory placed in the client with a prefixed name. Everything will be done hidden at the background with no user attantion required. FTP, HTTTP, SMTP/POP3, NNTP are all capable of transmit file and they are standard protocols based on TCP sockets. You may also design your own protocol based on your own requirements which are... -- I actualy have the protocol which is just the command based and everything works real good and smooth but sending and receiving files is once more something which I don't know if possible to do in the same way. Like: Server sends command: Get me the log file and client answers by sending the file, or server sends the setup.exe file and client responds by installing it. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
FTP, HTTTP, SMTP/POP3, NNTP are all capable of transmit file and they are standard protocols based on TCP sockets. You may also design your own protocol based on your own requirements which are... -- I actualy have the protocol which is just the command based and everything works real good and smooth but sending and receiving files is once more something which I don't know if possible to do in the same way. You can easily add file transfer to your protocol. For example, you implement a command file taking as argument the file size, filename and timestamp (UTC). Then the sender read the file block by block and send exactly the number of byte, then the command is finished. At receiver side, whe the file command is received, the receiver parse file size, timestamp and file namne, then switch linemode to off and wait the receive exatcly filesize bytes, save it in the given file (or stream) and then switch back linemode to on and send some OK message to the sender. Basically this is more or less how http 1.1 is working. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
Thank you :) Just what wanted to hear. With best regards -Original Message- From: Francois PIETTE Sent: Monday, February 14, 2011 2:19 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer TWSocketClient send receive files FTP, HTTTP, SMTP/POP3, NNTP are all capable of transmit file and they are standard protocols based on TCP sockets. You may also design your own protocol based on your own requirements which are... -- I actualy have the protocol which is just the command based and everything works real good and smooth but sending and receiving files is once more something which I don't know if possible to do in the same way. You can easily add file transfer to your protocol. For example, you implement a command file taking as argument the file size, filename and timestamp (UTC). Then the sender read the file block by block and send exactly the number of byte, then the command is finished. At receiver side, whe the file command is received, the receiver parse file size, timestamp and file namne, then switch linemode to off and wait the receive exatcly filesize bytes, save it in the given file (or stream) and then switch back linemode to on and send some OK message to the sender. Basically this is more or less how http 1.1 is working. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer TWSocketClient send receive files
Just what wanted to hear. My pleasure :-) You alos have the option to send the file in line mode and then you must encode each block for example using base64 (See OverbyteIcsMimeUtils). This is how SMTP/POP3/NNTP is handling files. This will takes more space but will offer pure line oriented protocol which you may prefer... or not. It's a matter of taste. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be - Original Message - From: daniel cc dan...@signedsource.com To: ICS support mailing twsocket@elists.org Sent: Monday, February 14, 2011 2:19 PM Subject: Re: [twsocket] TWSocketServer TWSocketClient send receive files Thank you :) Just what wanted to hear. With best regards -Original Message- From: Francois PIETTE Sent: Monday, February 14, 2011 2:19 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer TWSocketClient send receive files FTP, HTTTP, SMTP/POP3, NNTP are all capable of transmit file and they are standard protocols based on TCP sockets. You may also design your own protocol based on your own requirements which are... -- I actualy have the protocol which is just the command based and everything works real good and smooth but sending and receiving files is once more something which I don't know if possible to do in the same way. You can easily add file transfer to your protocol. For example, you implement a command file taking as argument the file size, filename and timestamp (UTC). Then the sender read the file block by block and send exactly the number of byte, then the command is finished. At receiver side, whe the file command is received, the receiver parse file size, timestamp and file namne, then switch linemode to off and wait the receive exatcly filesize bytes, save it in the given file (or stream) and then switch back linemode to on and send some OK message to the sender. Basically this is more or less how http 1.1 is working. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: client not removed from server list on disconnect
Hello Anton! destructor TSrvClient.Destroy; begin FreeAndNil(FList); // *** set breakpoint here to ensure client is destroyed *** end; This should be: destructor TSrvClient.Destroy; begin FreeAndNil(FList); inherited; // -- added call to ancestor end; Regards, Tobias -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: client not removed from server list on disconnect
Tobias, oh, thanks a lot! Today is the day when I use to forget to call ancestors' methods :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: client not removed from server list on disconnect
Probably because it is automatic in C++(?) Regards, SZ On Tue, Feb 1, 2011 at 5:53 PM, Anton S. an...@rambler.ru wrote: Tobias, oh, thanks a lot! Today is the day when I use to forget to call ancestors' methods :) -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT wrote: Man in the Middle attacks don't work if the man in the middle don't know how to handle the encrypted data/protocol he is intercepting. True, and how do you manage that is not happening? Can't be happening because the man in the middle can't generate valid data, or alter intercepted data maintaining its validity, if he can't break the encrypt algorithm in time to inject his packets of data. With a stolen key that's easy. This is also valid for SSL. No, the difference is that SSL is able to detect the man in the middle. Usually the certificate includes some information like the domain name or IP address, so even if the attacker used a stolen certificate peer verification would fail and the connection won't be established. Closed standards are inheritable much more secure than open standards. That's nothing but security through obscurity: http://en.wikipedia.org/wiki/Security_through_obscurity That's just a theoretic argument, not an undoubted reality. There are enough examples mentioned in that wiki article to prove the opposite. In this type of projects the use of the of this standard is wrong. He just don't need the SSL implementation complexity, nor the result slow to start communication, just to get his data secure. I don't know what _he needs, if _you want to invent your own security standards feel free to do so. SSL/TLS is used and accepted world-wide. Neither do I, but I'm assuming he only need what a generic data communication service needs in terms of security. Pass data in a way it can't be tampered/understood, if intercepted by someone outside the communication points. I'm not replying to you, Arno, to be impertinent. Far from that. It's just my opinion that a symmetric keyed algorithm, such as AES or Blowfish, with a clever time volatile salt added to the key, is enough for this case in particular. The weak point here is key delivery. Keys should be changed very frequently. How do you make sure that keys are not stolen and received by the right people? They should never be hard coded in the application. SSL negotiates a unique symmetric key per session, so even if the key was found by brute force it can be used only to decrypt a single SSL session. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
It is a very simple monitoring system. SSL is okay for this as far as I am concerned. As others have say to you already, just concentrate your efforts in the development of the client and server code. Leave the data encryption to the last stage of the project. Starting with the SSL components, just because you want your data exchange to have some kind of protection in the final implementation, is wrong. Start it simple, and increase the complexity after you are comfortable with what you have already. For me the mosty important here is to learn one thing at a time. So, leave SSL apart for now. If I listen you, I have to start learning something else also. Your way of handling is not going to help me to understand the ICS sockets better it will make things more complicated and difficult. Where I said that. It's exactly the contrary. You are the one that insist in the usage of a complex component from the start. A question, Would you help me with few code examples if I do exactly what you are saying? You have all you need to start in the ICS standard samples, and Francois and others already pointed you to the correct ones, and even gave you code examples to specific questions, but you continue asking the same questions, over and over again. My feel is that your knowledge of the Delphi language itself is very limited, and this is not the right place to learn it. If you really want to help me, Post me few examples (don't explain) of how to? - Send command to the client socket by using it's computer name It was been explained to you already, more than one time. - Recieve WMI information from the client socket == I am not asking you to help with WMI, I can do it myself, all I need is how to connect to WMI from server socket to the client socket and receive the data. Already explained too. Use the ongoing client to server connection to send data to the client. - How to handle the errors of sockets (example: if client loses the connection it gives error in the window, how this can be handled in the sockets). Just ask this directly in the list. I'm not an expert in the subject. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
With a stolen key that's easy. Sure, and this is exactly what SSL try to circumvent. But not so easy if the encrypt key is not a fixed value, but a variable one. The attacker will need to stole the client or server code and reverse engineering it too. This is also valid for SSL. No, the difference is that SSL is able to detect the man in the middle. Usually the certificate includes some information like the domain name or IP address, so even if the attacker used a stolen certificate peer verification would fail and the connection won't be established. Man in the middle attacks intercepts data in a transparent way, in the middle of the line and in a ongoing communication . The in the middle IP address is not even a variable for the peer verification. I'm not replying to you, Arno, to be impertinent. Far from that. It's just my opinion that a symmetric keyed algorithm, such as AES or Blowfish, with a clever time volatile salt added to the key, is enough for this case in particular. The weak point here is key delivery. Keys should be changed very frequently. How do you make sure that keys are not stolen and received by the right people? They should never be hard coded in the application. SSL negotiates a unique symmetric key per session, so even if the key was found by brute force it can be used only to decrypt a single SSL session. True, but you can also have you own key exchange method too. And you would reply, so why not use the already available SSL protocol that do exactly that? Because everyone know how it works, and if I'm going to develop my Client and Server, I don't need to use something that is public available. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
So, leave SSL apart for now. I don't see any reasons to leave the SSL because I already spent 3 weeks with making lot of tests and have learned much about SSL and I will continue. I am also hoping that you guys try to understand my approach instead of asking to leave it. SSL is good enough for me as Arno also has convinced me about it. If I can figure out few things than I will have no problems of doing my project to the end because I am already very close to it. My feel is that your knowledge of the Delphi language itself is very limited, and this is not the right place to learn it. This is something which isn't easy to tell about what is the limited knowledge. Every programmer is good in some fields and the network communication is not mine. It was been explained to you already, more than one time. You probably have which I haven't understand i bit of it. Explaining is one thing and understanding is another. And expecting that everything is as simple as you have explained and expecting that the person is capable of understanding exactly what you are saying is also another thing. Everything will work perfectly when you guys stop treating me as a guru and try to give me samples of what can be done and how it can be used. Please MARK ON THIS!! I am not asking anyone to write code for me, I can do that myself. All I am asking is explain it with a sample of what can be done and how can be done, guide me to the samples, guide me to the direction. I am most of the time getting one explanation and 4 different directions such as yours telling me to use something else than what I am trying to do. And. Please let's quit this here and continue to some other subjects. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
SOLVED!! -Original Message- From: daniel cc Sent: Thursday, January 27, 2011 7:33 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer and TWSocket Port So, leave SSL apart for now. I don't see any reasons to leave the SSL because I already spent 3 weeks with making lot of tests and have learned much about SSL and I will continue. I am also hoping that you guys try to understand my approach instead of asking to leave it. SSL is good enough for me as Arno also has convinced me about it. If I can figure out few things than I will have no problems of doing my project to the end because I am already very close to it. My feel is that your knowledge of the Delphi language itself is very limited, and this is not the right place to learn it. This is something which isn't easy to tell about what is the limited knowledge. Every programmer is good in some fields and the network communication is not mine. It was been explained to you already, more than one time. You probably have which I haven't understand i bit of it. Explaining is one thing and understanding is another. And expecting that everything is as simple as you have explained and expecting that the person is capable of understanding exactly what you are saying is also another thing. Everything will work perfectly when you guys stop treating me as a guru and try to give me samples of what can be done and how it can be used. Please MARK ON THIS!! I am not asking anyone to write code for me, I can do that myself. All I am asking is explain it with a sample of what can be done and how can be done, guide me to the samples, guide me to the direction. I am most of the time getting one explanation and 4 different directions such as yours telling me to use something else than what I am trying to do. And. Please let's quit this here and continue to some other subjects. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT wrote: With a stolen key that's easy. Sure, and this is exactly what SSL try to circumvent. But not so easy if the encrypt key is not a fixed value, but a variable one. The attacker will need to stole the client or server code and reverse engineering it too. This is also valid for SSL. No, the difference is that SSL is able to detect the man in the middle. Usually the certificate includes some information like the domain name or IP address, so even if the attacker used a stolen certificate peer verification would fail and the connection won't be established. Man in the middle attacks intercepts data in a transparent way, in the middle of the line and in a ongoing communication . The in the middle IP address is not even a variable for the peer verification. Without the certificate(s) and private key(s) he may intercept transparently as long as he likes. When he wants to decrypt the session on the fly he has to go thru the handshake process on behave of the victim by presenting the stolen certificate(s), acting as a proxy server. I'm not replying to you, Arno, to be impertinent. Far from that. It's just my opinion that a symmetric keyed algorithm, such as AES or Blowfish, with a clever time volatile salt added to the key, is enough for this case in particular. The weak point here is key delivery. Keys should be changed very frequently. How do you make sure that keys are not stolen and received by the right people? They should never be hard coded in the application. SSL negotiates a unique symmetric key per session, so even if the key was found by brute force it can be used only to decrypt a single SSL session. True, but you can also have you own key exchange method too. And you would reply, so why not use the already available SSL protocol that do exactly that? Exactly. Because everyone know how it works, That's the point, bugs in proprietary protocols are usually not found as fast as bugs in public protocols (by the good guys). and if I'm going to develop my Client and Server, I don't need to use something that is public available. Of course you can, I just doubt that it's more secure than properly implemented SSL/TLS. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
On 27-01-2011 18:27, Arno Garrels wrote: Without the certificate(s) and private key(s) he may intercept transparently as long as he likes. When he wants to decrypt the session on the fly he has to go thru the handshake process on behave of the victim by presenting the stolen certificate(s), acting as a proxy server. But stolen keys is your previous argument. After you have it, decoding the data is much more easy if you know how the key is used to encrypt/decrypt it. Knowing the encrypt algorithm is also important to, with brute-force, decode the data, if you just want to see what the communication is all about. That's why the security agencies don't like closed protocols. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT wrote: That's why the security agencies don't like closed protocols. Nobody knows whether a proprietary security protocol has some built-in universal key, given i.e. to a security agency. That's also why users should not rely on proprietary security protocols and cryptographic algorithms. AFAIK, serious companies won't buy such software, perhaps except they are big enough to verify from the source code that it's safe to use. So using open, well known standards is a selling point too, at least in the league I'm playing. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
I would like to use the same port which the site A uses when it connects to the site B. Yes, you can have a same listening port at both sides. -- francois.pie...@overbyte.be http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
I would like to use the same port which the site A uses when it connects to the site B. Yes, you can have a same listening port at both sides. Thanks for the response. Yes, I know I can listen same ports but my question is, can I do that at Site A without opening any ports at the routers/firewalls? The goal is to have ports open only at site B which is the server. I am I am explaining this correct. Thanks -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
I would like to use the same port which the site A uses when it connects to the site B. Yes, you can have a same listening port at both sides. Thanks for the response. Yes, I know I can listen same ports but my question is, can I do that at Site A without opening any ports at the routers/firewalls? As soon as you have a listening port on a LAN accessible from the outside thru a firewall or NAT router, you have to open the port. The goal is to have ports open only at site B which is the server. The use only one socket and make all communications over that unique connection. No other choise in your situation. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
The use only one socket and make all communications over that unique connection. No other choise in your situation. Thanks Francois, Please do correct me if I am wrong. Now, You are saying that it is not possible to have only server port opened, I need to open port 443 in both sites. Site A port 443 need to be open from PC and from the firewall. Site B port 443 need to be open from PC and from firewall. If this is correct than, means there are no options to reverse the communication between the sockets and also means the client pc cannot be accessed in any ways. Means also, I cannot run remote client windows Command prompt from the server. Means also, I cannot use WMI to collect the client system info or modify the client system info. I am just hoping that I am totally wrong!! -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
Please do correct me if I am wrong. Now, You are saying that it is not possible to have only server port opened, I need to open port 443 in both sites. Site A port 443 need to be open from PC and from the firewall. Site B port 443 need to be open from PC and from firewall. No, I didn't meant to say that. I said: You can have a listing port 443 in Site A and/or in Side B as you like. As soon as you have a listening port on any side, then to be able to reach that port from the outside, you have to open the port in the firewall/router/whatever on the box which is on the physical link and also on the computer itself. This is not related to ICS but simply how TCP/IP networking works. If this is correct than, means there are no options to reverse the communication between the sockets and also means the client pc cannot be accessed in any ways. A client PC can never been reached. This is exactly what make the difference between a client and a server. A server is the one accepting incomming connections. The client is the one initiating outgoing connections. As far as firewall/routing is concerned, whatever the direction of a connection, the firewall/router must be configured to accept the connection. Usually all outgoing connections are open and all incomming connections are closed. Means also, I cannot run remote client windows Command prompt from the server. Not correct. The client can open the connection with the server. Then the server may send commands to the client, using the established connection, to instruct the client to run a command prompt (with appropriate I/O redirection so that input/output goes from/to the connection between client and server. Means also, I cannot use WMI to collect the client system info or modify the client system info. Not correct. I am just hoping that I am totally wrong!! Almost. I think you have not figured the software layers in your head. One thing is the link layer (the connection between client and server). Another think is the application layer (commands/responses transported by the link, in both direction) -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
(I am using SSL sockets in both sites) Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? You will be much more secure if you encode your data, with your own method, using a much more powerful encrypt algorithm than the used by SSL. And you even get ride of third-party code, such as the OpenSSL DLLs. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT wrote: (I am using SSL sockets in both sites) Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? You will be much more secure if you encode your data, with your own method, using a much more powerful encrypt algorithm than the used by SSL. SSL/TLS aktually uses common, powerful and strong encryption algorithms. However secure peer to peer communication is much more than just that. SSL security includes, for instance, peer verification and protects against Man in the Middle attacks. http://en.wikipedia.org/wiki/Transport_Layer_Security http://en.wikipedia.org/wiki/Public-key_cryptography -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
(I am using SSL sockets in both sites) Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? You will be much more secure if you encode your data, with your own method, using a much more powerful encrypt algorithm than the used by SSL. And you even get ride of third-party code, such as the OpenSSL DLLs. -- I Insist because I have no other options because I don't have knowledge of doing it in the other way. Is that clear enough for you? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
Thanks Arno, Your answer is just a music to my ears :) RTT wrote: (I am using SSL sockets in both sites) Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? You will be much more secure if you encode your data, with your own method, using a much more powerful encrypt algorithm than the used by SSL. SSL/TLS aktually uses common, powerful and strong encryption algorithms. However secure peer to peer communication is much more than just that. SSL security includes, for instance, peer verification and protects against Man in the Middle attacks. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
SSL/TLS aktually uses common, powerful and strong encryption algorithms. However secure peer to peer communication is much more than just that. SSL security includes, for instance, peer verification and protects against Man in the Middle attacks. Man in the Middle attacks don't work if the man in the middle don't know how to handle the encrypted data/protocol he is intercepting. Closed standards are inheritable much more secure than open standards. In this type of projects the use of the of this standard is wrong. He just don't need the SSL implementation complexity, nor the result slow to start communication, just to get his data secure. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT wrote: SSL/TLS aktually uses common, powerful and strong encryption algorithms. However secure peer to peer communication is much more than just that. SSL security includes, for instance, peer verification and protects against Man in the Middle attacks. Man in the Middle attacks don't work if the man in the middle don't know how to handle the encrypted data/protocol he is intercepting. True, and how do you manage that is not happening? Closed standards are inheritable much more secure than open standards. That's nothing but security through obscurity: http://en.wikipedia.org/wiki/Security_through_obscurity In this type of projects the use of the of this standard is wrong. He just don't need the SSL implementation complexity, nor the result slow to start communication, just to get his data secure. I don't know what _he needs, if _you want to invent your own security standards feel free to do so. SSL/TLS is used and accepted world-wide. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
Man in the Middle attacks don't work if the man in the middle don't know how to handle the encrypted data/protocol he is intercepting. True, and how do you manage that is not happening? Can't be happening because the man in the middle can't generate valid data, or alter intercepted data maintaining its validity, if he can't break the encrypt algorithm in time to inject his packets of data. This is also valid for SSL. Closed standards are inheritable much more secure than open standards. That's nothing but security through obscurity: http://en.wikipedia.org/wiki/Security_through_obscurity That's just a theoretic argument, not an undoubted reality. In this type of projects the use of the of this standard is wrong. He just don't need the SSL implementation complexity, nor the result slow to start communication, just to get his data secure. I don't know what _he needs, if _you want to invent your own security standards feel free to do so. SSL/TLS is used and accepted world-wide. Neither do I, but I'm assuming he only need what a generic data communication service needs in terms of security. Pass data in a way it can't be tampered/understood, if intercepted by someone outside the communication points. I'm not replying to you, Arno, to be impertinent. Far from that. It's just my opinion that a symmetric keyed algorithm, such as AES or Blowfish, with a clever time volatile salt added to the key, is enough for this case in particular. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT, First, I am not building a bank system where I would need extremely high security. It is a very simple monitoring system. SSL is okay for this as far as I am concerned. You may be right or may not. For me the mosty important here is to learn one thing at a time. If I listen you, I have to start learning something else also. Your way of handling is not going to help me to understand the ICS sockets better it will make things more complicated and difficult. A question, Would you help me with few code examples if I do exactly what you are saying? No!! I didn't think so. My motto is, If you aren't gonna help, don't bother because I already have got too many ideas and ways to use as the blowfish which you are talking about is one of them. Right now, I don't need ideas which makes things more difficult and complicated, I only need help for learning the ICS socket communication. If you really want to help me, Post me few examples (don't explain) of how to? - Send command to the client socket by using it's computer name - Recieve WMI information from the client socket == I am not asking you to help with WMI, I can do it myself, all I need is how to connect to WMI from server socket to the client socket and receive the data. - How to handle the errors of sockets (example: if client loses the connection it gives error in the window, how this can be handled in the sockets). This samples could be uploaded to the ICS component site where every starter would have possibility to use them as help source. Thanks -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and TWSocket Port
RTT: Why you insist in use SSL if the client and server applications are coded by you?!! Why use a standard that only exist because of the need to connect many different implementations of clients to many different implementations of servers? It's easy to implement encryption but you'll have to add key exchange also and exclude possibility of key sniffing what is quite harder. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port?
Arno Garrels wrote: Arno Garrels wrote: Back to the keyboard then. I think a found a way everybody can live with. Will commit this new attempt to the IPv6 branch when I think it's alpha-ready. Then we are able to continue the discussion based on an existing implementation. Done, IPv6 branch updated (new multi-listen feature in TWSocketServer and descendants). Francois, I realy appreciated if you took a look at it. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port?
Probably breaking changes could be iontroduced the day ICS stops supporting non unicode compiler. Probably ICS-V8 could be that one, maybe to be released at the same time as Delphi crossplatfom, abandonning support for most of old Delphi version. I'm thinking about letting ICS-V7 live for D7-D2010 and have ICS-V8 for Delphi-XE and the next one. Just ideas right now. Oh, maybe that will be a good occasion for some cleanup and slight code redesign. And maybe these changes will include more clear FTPcli error reporting (the thing I wrote about several months ago). -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port?
Probably breaking changes could be iontroduced the day ICS stops supporting non unicode compiler. Probably ICS-V8 could be that one, maybe to be released at the same time as Delphi crossplatfom, abandonning support for most of old Delphi version. I'm thinking about letting ICS-V7 live for D7-D2010 and have ICS-V8 for Delphi-XE and the next one. Just ideas right now. Oh, maybe that will be a good occasion for some cleanup and slight code redesign. Sure ! And also to benefit from new construct in the language such as generics and others. And maybe these changes will include more clear FTPcli error reporting (the thing I wrote about several months ago). Maybe. Designing V8 will be a long process. Every option is still open. My dream is to have a single source code for Windows, MAC and Linux. This probably means ICS-V8 will looks more like ICS-for-Kylix than current ICS version. This is because MAC OSX and Linux do not work with messages as Windows do. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port?
Arno Garrels wrote: Back to the keyboard then. I think a found a way everybody can live with. Will commit this new attempt to the IPv6 branch when I think it's alpha-ready. Then we are able to continue the discussion based on an existing implementation. Done, IPv6 branch updated (new multi-listen feature in TWSocketServer and descendants). -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listeningtoseveralIP:Port?
Francois PIETTE wrote: Probably breaking changes could be iontroduced the day ICS stops supporting non unicode compiler. Probably ICS-V8 could be that one, maybe to be released at the same time as Delphi crossplatfom, abandonning support for most of old Delphi version. I'm thinking about letting ICS-V7 live for D7-D2010 and have ICS-V8 for Delphi-XE and the next one. Just ideas right now. Oh, maybe that will be a good occasion for some cleanup and slight code redesign. Sure ! And also to benefit from new construct in the language such as generics and others. And maybe these changes will include more clear FTPcli error reporting (the thing I wrote about several months ago). Maybe. Designing V8 will be a long process. Every option is still open. My dream is to have a single source code for Windows, MAC and Linux. This probably means ICS-V8 will looks more like ICS-for-Kylix than current ICS version. This is because MAC OSX and Linux do not work with messages as Windows do. IMO emulating windows messages on the MAC and Linux is just a workaround, if not a TRUE hack ;) -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how tomakeitlisteningtoseveralIP:Port?
Francois PIETTE wrote: In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. That is a good idea. This is not possible to achive with current TWSocketServer design. I wouldn't be so sure. In the meanwhile I think I got the idea. A new very small class TWServerSocket that is instantiated and registered by the TWSocketServer for each new listening socket. TWServerSocket.Listen calls WSAASyncSelect with TWSocketServer's window handle. Small change, not a small new class but a TWSocketServer and currently I override WMASyncSelect and Accept, works like a charm. The MultiListenSockets are a TCollection descendant so accessible in OI as well. The only limit is that they (the additional listening sockets) must run in the same thread context as their owner TWSocketServer. If that wasn't the case an assertion is raised. The idea of a small class is a good idea altough it defenitely force al events to be shared between all listening sockets. That's not a problem imo but you raised the issue previously. Back to the keyboard then. I think a found a way everybody can live with. Will commit this new attempt to the IPv6 branch when I think it's alpha-ready. Then we are able to continue the discussion based on an existing implementation. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make itlisteningtoseveralIP:Port?
Arno Garrels wrote: Francois PIETTE wrote: In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. That is a good idea. This is not possible to achive with current TWSocketServer design. I wouldn't be so sure. In the meanwhile I think I got the idea. A new very small class TWServerSocket that is instantiated and registered by the TWSocketServer for each new listening socket. TWServerSocket.Listen calls WSAASyncSelect with TWSocketServer's window handle. Small change, not a small new class but a TWSocketServer and currently I override WMASyncSelect and Accept, works like a charm. The MultiListenSockets are a TCollection descendant so accessible in OI as well. The only limit is that they (the additional listening sockets) must run in the same thread context as their owner TWSocketServer. If that wasn't the case an assertion is raised. Means you have to Add new listeners in the same thread context and you may not attach the main instance to another thread context when there are MultiListenSockets in the collection, what do you think? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to makeitlisteningtoseveralIP:Port?
In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. That is a good idea. This is not possible to achive with current TWSocketServer design. I wouldn't be so sure. In the meanwhile I think I got the idea. A new very small class TWServerSocket that is instantiated and registered by the TWSocketServer for each new listening socket. TWServerSocket.Listen calls WSAASyncSelect with TWSocketServer's window handle. Small change, not a small new class but a TWSocketServer and currently I override WMASyncSelect and Accept, works like a charm. The MultiListenSockets are a TCollection descendant so accessible in OI as well. The only limit is that they (the additional listening sockets) must run in the same thread context as their owner TWSocketServer. If that wasn't the case an assertion is raised. The idea of a small class is a good idea altough it defenitely force al events to be shared between all listening sockets. That's not a problem imo but you raised the issue previously. Means you have to Add new listeners in the same thread context and you may not attach the main instance to another thread context when there are MultiListenSockets in the collection, what do you think? Adding new listeners from another thread is not a problem if a custom message is posted so that the code (the message handler) creating the new listener is executed in the TWSocketServer thread whatever the originating thread is. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listeningtoseveralIP:Port ?
Francois PIETTE wrote: This design would not break any existing code and would ease using the new multiple listen easy to implement in higher level components. It won't break existing code, but remains a hack IMO. I don't agree with you. Please explain. The main instance keep track of the secondary instances. Yes, that should be a TComponentList owning the instances. A new r/o property MultiListenList (and btw MultiListenListCount) would gives the list of all TWSocketServer, in the order specified in the Addr property. All TWSocketServer share the same list. MultiListenIndex would be -1, MultiListenList would be nil and MultiListenCount would be 0 if TWSocketServer is used alone. Each get a new property MultiListenIndex which is their index in that list. At first I would not expose the list but provide public methods to Add and Delete instances and to access instances by index (r/o). I would simply fake the list with count 1 when there are no additional instances and the list is not assigned. Listener[0] returned Self, ListenerCount 1 etc.. All clients should be added to main instance's client list, the Sender argument in server events gives you the server object triggered the event. I'm not sure whether you have that in mind, have you? All event handlers are common to all TWSocketServer instances. That should be the default assignment after a call to method Add() only to ease setup of the new instance. All event assignments and unassignments have to be kept in sync. They will be easily with setter and getter. That may work, but it is confusing since one can easily change the event handlers of additional instances. It was cleaner to not synchronize event assignments except in method AddServer() to set the defaults. Also I would not bind the construction/destruction of the additional instances to the Addr string property (grrr), Why grrr ? Please explain. At least that should be a new string property since Addr and Port identify the instance too. It remains a hack as long as not all relevant persistent properties are available in Object Inspector. What about SslEnable? It's cleaner to let users explicitly create additional instances at runtime. If just a fraction of persistent properties are assigned the component user has to set the rest at runtime, that's confusing. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make itlisteningtoseveralIP:Port ?
Hi Arno, After your first message I tough you where lighting years away from my design. I understand now that we agree on most items. About the list: I had not expressed how the list has to be implemented. The requirement is that the component offer a method (in general sense, not in object pascal syntax) for the developper to iterate thru all subcomponent. An indexed property has my favor. We should propably think ahead when D7 has no more ot be supported and provide a design which will easily allow an iteration implementation to use foreach construct. Methods to add/remove subcomponents are OK. No problem About event handler: We have the choice to enforce all are in sync or not. I'm not sure yet what is best. At least it is easy to propagate all event handlers to all component whichever subcomponent is used to assign the new event handler. It is also very easy to have all subcomponent event handler created equal to the main and let the user change it later. Don't know yet what is better. About Addr/Port: I agree with you. A new property MultiListenAddrPort has to be used and all instance will get his own value assigned to his Addr and Port properties. About persistent properties: I don't understand the issue. About general design consideration: My idea is to create a component which would cover 80% of the needs for multiple listening sockets. All very special cases could still be handled as it is today with individual components. Trying to make the new component fit 100% of the needs will result in a too complex component. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be - Original Message - From: Arno Garrels arno.garr...@gmx.de To: ICS support mailing twsocket@elists.org Sent: Saturday, November 27, 2010 12:01 PM Subject: Re: [twsocket] TWSocketServer: how to make itlisteningtoseveralIP:Port ? Francois PIETTE wrote: This design would not break any existing code and would ease using the new multiple listen easy to implement in higher level components. It won't break existing code, but remains a hack IMO. I don't agree with you. Please explain. The main instance keep track of the secondary instances. Yes, that should be a TComponentList owning the instances. A new r/o property MultiListenList (and btw MultiListenListCount) would gives the list of all TWSocketServer, in the order specified in the Addr property. All TWSocketServer share the same list. MultiListenIndex would be -1, MultiListenList would be nil and MultiListenCount would be 0 if TWSocketServer is used alone. Each get a new property MultiListenIndex which is their index in that list. At first I would not expose the list but provide public methods to Add and Delete instances and to access instances by index (r/o). I would simply fake the list with count 1 when there are no additional instances and the list is not assigned. Listener[0] returned Self, ListenerCount 1 etc.. All clients should be added to main instance's client list, the Sender argument in server events gives you the server object triggered the event. I'm not sure whether you have that in mind, have you? All event handlers are common to all TWSocketServer instances. That should be the default assignment after a call to method Add() only to ease setup of the new instance. All event assignments and unassignments have to be kept in sync. They will be easily with setter and getter. That may work, but it is confusing since one can easily change the event handlers of additional instances. It was cleaner to not synchronize event assignments except in method AddServer() to set the defaults. Also I would not bind the construction/destruction of the additional instances to the Addr string property (grrr), Why grrr ? Please explain. At least that should be a new string property since Addr and Port identify the instance too. It remains a hack as long as not all relevant persistent properties are available in Object Inspector. What about SslEnable? It's cleaner to let users explicitly create additional instances at runtime. If just a fraction of persistent properties are assigned the component user has to set the rest at runtime, that's confusing. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to makeitlisteningtoseveralIP:Port ?
Francois PIETTE wrote: About event handler: We have the choice to enforce all are in sync or not. I'm not sure yet what is best. At least it is easy to propagate all event handlers to all component whichever subcomponent is used to assign the new event handler. It is also very easy to have all subcomponent event handler created equal to the main and let the user change it later. Don't know yet what is better. The latter is clearly better IMO. Otherwise it's not flexible enough and also requires that changing the event handler of any instance has to propagate to any other instance. About Addr/Port: I agree with you. A new property MultiListenAddrPort has to be used and all instance will get his own value assigned to his Addr and Port properties. About persistent properties: I don't understand the issue. There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Steps required were iterate over the list of instances, find the one with Port 443 and set SslEnabled and perhaps SslMode as well. I also wonder when the instances specified by property MultiListenAddrPort will be created? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how tomakeitlisteningtoseveralIP:Port ?
There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Maybe not. Look at TStatusBar, you can add panels and set properties for each of them individually. This is the kind of thing we need. I also wonder when the instances specified by property MultiListenAddrPort will be created? To be able to have the behaviour I described above, the instances must be created/destroyed on the fly when MultiListenAddrPort is assigned a value. We should do just a the panels in TStatusBar: have a design time user interface (property editor for MultiListenAddrPort) allowing the user to create each instance and assign individual addr and port values. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port ?
Francois PIETTE wrote: There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Maybe not. Look at TStatusBar, you can add panels and set properties for each of them individually. This is the kind of thing we need. Sure it is possible with TCollection and TCollectionItem. Now we come closer and closer to my first multi-listen server I wrote 2 years ago g. with TCollection however a new string property MultiListenAddrPort was superfluous as well since all properties can the be set with OI or at run time. I also wonder when the instances specified by property MultiListenAddrPort will be created? To be able to have the behaviour I described above, the instances must be created/destroyed on the fly when MultiListenAddrPort is assigned a value. We should do just a the panels in TStatusBar: have a design time user interface (property editor for MultiListenAddrPort) allowing the user to create each instance and assign individual addr and port values. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. I think you should give that a trial yourself, I cannot see what you want to achieve exactly. Would your design have a single client list or is it required to iterate over multiple different client lists? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to several IP:Port?
A string property is very easy for the component user at runtime. A single line of code allows to setup all the subcomponents. And this ease saving/loading the configuration for the application. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. Properties assigned thru the OI are not set for each character typed by the user, but at the end of the input. The setter is called once. Having a component reactying to user input thru the OI is what's makes Delphi being what it is. I don't see any problem creating/destroying objects during this process. We are at design time and the component will almost do nothing. -- francois.pie...@overbyte.be http://www.overbyte.be - Original Message - From: Arno Garrels arno.garr...@gmx.de To: ICS support mailing twsocket@elists.org Sent: Saturday, November 27, 2010 4:58 PM Subject: Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port? Francois PIETTE wrote: There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Maybe not. Look at TStatusBar, you can add panels and set properties for each of them individually. This is the kind of thing we need. Sure it is possible with TCollection and TCollectionItem. Now we come closer and closer to my first multi-listen server I wrote 2 years ago g. with TCollection however a new string property MultiListenAddrPort was superfluous as well since all properties can the be set with OI or at run time. I also wonder when the instances specified by property MultiListenAddrPort will be created? To be able to have the behaviour I described above, the instances must be created/destroyed on the fly when MultiListenAddrPort is assigned a value. We should do just a the panels in TStatusBar: have a design time user interface (property editor for MultiListenAddrPort) allowing the user to create each instance and assign individual addr and port values. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. I think you should give that a trial yourself, I cannot see what you want to achieve exactly. Would your design have a single client list or is it required to iterate over multiple different client lists? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to severalIP:Port?
Francois PIETTE wrote: A string property is very easy for the component user at runtime. A single line of code allows to setup all the subcomponents. And this ease saving/loading the configuration for the application. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. Properties assigned thru the OI are not set for each character typed by the user, but at the end of the input. The setter is called once. Having a component reactying to user input thru the OI is what's makes Delphi being what it is. I don't see any problem creating/destroying objects during this process. We are at design time and the component will almost do nothing. So, as I understand the approach the TWSocketServer shall be extended as a wrapper or helper to make the workaround of creating multiple TWSocketServer instances in code easier. In this case it _might be better to create a new component that handles this? In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. This is not possible to achive with current TWSocketServer design. Introducing new workarounds (hacks) should be avoided, instead in future versions it should be allowed to add some breaking changes and to fix current design flaw (my current IPv6 multi-listening hack should be removed from the IPv6 branche as well for the same reason). -- Arno Garrels -- francois.pie...@overbyte.be http://www.overbyte.be - Original Message - From: Arno Garrels arno.garr...@gmx.de To: ICS support mailing twsocket@elists.org Sent: Saturday, November 27, 2010 4:58 PM Subject: Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port? Francois PIETTE wrote: There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Maybe not. Look at TStatusBar, you can add panels and set properties for each of them individually. This is the kind of thing we need. Sure it is possible with TCollection and TCollectionItem. Now we come closer and closer to my first multi-listen server I wrote 2 years ago g. with TCollection however a new string property MultiListenAddrPort was superfluous as well since all properties can the be set with OI or at run time. I also wonder when the instances specified by property MultiListenAddrPort will be created? To be able to have the behaviour I described above, the instances must be created/destroyed on the fly when MultiListenAddrPort is assigned a value. We should do just a the panels in TStatusBar: have a design time user interface (property editor for MultiListenAddrPort) allowing the user to create each instance and assign individual addr and port values. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. I think you should give that a trial yourself, I cannot see what you want to achieve exactly. Would your design have a single client list or is it required to iterate over multiple different client lists? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port?
So, as I understand the approach the TWSocketServer shall be extended as a wrapper or helper to make the workaround of creating multiple TWSocketServer instances in code easier. In this case it _might be better to create a new component that handles this? I think most server application can be enhanced easily by upgrading ICS to a version with and enhanced TWSocketServer for multilistening. Most server applications (or component) just want to listen on several IP:Port and nothing else changed. The goal is to make that case (Pareto law: 80% are of this type 20% are more complex) as easy as possible. That's why at first glance I tough about the addr property. Side note: you objected about the addr property because it participate to the component identity. I'm just thinking is is enough to add one more component. The main component would never listen to anything, instead, as much subcomponents would be created as there are IP:Port to listen to. This makes the design more orthogonal. Handling the main component would affect all subcomponent. Handling a subcomponent would affect only that one. In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. That is a good idea. This is not possible to achive with current TWSocketServer design. I wouldn't be so sure. instead in future versions it should be allowed to add some breaking changes and to fix current design flaw (my current IPv6 multi-listening hack should be removed from the IPv6 branche as well for the same reason). One of the ICS major key success factor has always been the rule don't break existing code. I would like to avoid as much as possible breaking changes. Probably breaking changes could be iontroduced the day ICS stops supporting non unicode compiler. Probably ICS-V8 could be that one, maybe to be released at the same time as Delphi crossplatfom, abandonning support for most of old Delphi version. I'm thinking about letting ICS-V7 live for D7-D2010 and have ICS-V8 for Delphi-XE and the next one. Just ideas right now. -- francois.pie...@overbyte.be http://www.overbyte.be - Original Message - From: Arno Garrels arno.garr...@gmx.de To: ICS support mailing twsocket@elists.org Sent: Saturday, November 27, 2010 6:32 PM Subject: Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port? Francois PIETTE wrote: A string property is very easy for the component user at runtime. A single line of code allows to setup all the subcomponents. And this ease saving/loading the configuration for the application. As I said before creating/destroying objects on the fly depending on what the user types in a string property sounds bad in my ears. Properties assigned thru the OI are not set for each character typed by the user, but at the end of the input. The setter is called once. Having a component reactying to user input thru the OI is what's makes Delphi being what it is. I don't see any problem creating/destroying objects during this process. We are at design time and the component will almost do nothing. So, as I understand the approach the TWSocketServer shall be extended as a wrapper or helper to make the workaround of creating multiple TWSocketServer instances in code easier. In this case it _might be better to create a new component that handles this? In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. This is not possible to achive with current TWSocketServer design. Introducing new workarounds (hacks) should be avoided, instead in future versions it should be allowed to add some breaking changes and to fix current design flaw (my current IPv6 multi-listening hack should be removed from the IPv6 branche as well for the same reason). -- Arno Garrels -- francois.pie...@overbyte.be http://www.overbyte.be - Original Message - From: Arno Garrels arno.garr...@gmx.de To: ICS support mailing twsocket@elists.org Sent: Saturday, November 27, 2010 4:58 PM Subject: Re: [twsocket] TWSocketServer: howtomakeitlisteningtoseveralIP:Port? Francois PIETTE wrote: There is, for instance, property SslEnable in TWSocketServer descendant TSslWSocketServer that is propagated to clients. Let's say you want two listeners /192.168.1.1:80/192.168.1.1:443 one SSL and one plain text. Then you have to set SslEnable property at run time. Maybe not. Look at TStatusBar, you can add panels and set properties for each of them individually. This is the kind of thing we need. Sure it is possible with TCollection and TCollectionItem. Now we come closer and closer to my first multi-listen server I wrote 2 years ago g. with TCollection however a new string property MultiListenAddrPort was superfluous as well since all properties can the be set with OI or at run time. I also
Re: [twsocket] TWSocketServer: how to make it listening to
Hello! Now let's assume that application listens on two ip:port pairs, each for different kind of communication. For example, 192.168.1.1:2048 and 192.168.1.1:27015. Also, let's assume that user can define on which ip:port one of servers should listen (another ip:port pair would be fixed). What happens if user enters /192.168.1.1:2048/192.168.1.1:27015/ (for example, because he/she heard/read somewhere that with new ICS this is possible, and the application is based on ICS)? There would be clear conflict between either: - one of server components that now owns both ports and another that attempts to bind to owned port *or* - the second component that correctly owns 27015 port and first component that attempts to bind subservers to both 2048 and 27015 ports (depends on component creation order). This is not different than usual. Any application asking for to listen on a IP:port has to be ready to be refused because the port is already in use. Having multiple IP:port created by a single component doesn't change anything here. The application would handle it exactly the same way. Not quite, because application *doesn't know* that it's going to listen on three (or more) ports (instead of two). While that change doesn't directly break existing code, it might introduce difficult to understand bugs, as seen above. I don't agree with you. The design as I suggest make it totally equivalent to having multiple independant TWSocketServer component with common event handlers. As far as I understand your design well, it's not totally equivalent because it will treat any single bind failure as a total disaster, even if another binds are meant to be spare and actually not required to function (failure to bind on other ip:ports is not considered fatal). -- Piotr Dałek enigmati...@interia.pl Nie polegaj na drogowcach #8211; sam zaskocz zimę i ZMIEŃ OPONY! SprawdĹş http://linkint.pl/f286e -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listeningtoseveralIP:Port?
Francois PIETTE wrote: In my option a real ICS multi-listening server should handle all client objects in one list. These client objects should be aware of what listening socket accepted them. That is a good idea. This is not possible to achive with current TWSocketServer design. I wouldn't be so sure. In the meanwhile I think I got the idea. A new very small class TWServerSocket that is instantiated and registered by the TWSocketServer for each new listening socket. TWServerSocket.Listen calls WSAASyncSelect with TWSocketServer's window handle. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to several IP:Port ?
Hello Francois, Of course it will be a cool design but I do not see any need for multiple listening except 0.0.0.0 and ::0 as I have seen none of the servers doing it. Regards, SZ On Fri, Nov 26, 2010 at 11:12 AM, Francois PIETTE francois.pie...@skynet.be wrote: Hello all, Since the Addr property is a string, the component could easily handle any special format in that field to support multiple listening IP:Port (Interface to listen to and port to use on that interface). I think something like /192.168.1.1:80/0.0.0.0:81/. IPv4 or IPv6 could be used. The component can easily find out that there is a multiple IP:Port because of the / at the beginning. The port property would be used as default value when no prot is specified in the Addr for a given IP. As an example: Addr := /192.168.1.1/192.168.2.1/10.1.2.3:80/ and Port := 81 would make listening on port 81 for 192.168.1.1 and 192.168.2.1, and listening on port 80 for interface 10.1.2.3. The working would be this: The main TWSocketServer component listen on the first IP:Port specifyed and create secondary TWSocketServer instances to listen to the other IP:Port. The main instance keep track of the secondary instances. A new r/o property MultiListenList (and btw MultiListenListCount) would gives the list of all TWSocketServer, in the order specified in the Addr property. All TWSocketServer share the same list. Each get a new property MultiListenIndex which is their index in that list. All event handlers are common to all TWSocketServer instances. MultiListenIndex would be -1, MultiListenList would be nil and MultiListenCount would be 0 if TWSocketServer is used alone. If needed, but not sure it is, new methods could be added to handle all TWSocketServers at once: MultiListenListen, MultiListenClose and so on. This design would not break any existing code and would ease using the new multiple listen easy to implement in higher level components. What do you think ? -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to several IP:Port ?
I do not see any need for multiple listening except 0.0.0.0 and ::0 as I have seen none of the servers doing it. No commercial server listens on 0.0.0.0 since commercial servers all have multiple IP addresses with multiple users and multiple applications, with the exception of IIS that hogs all addresses even if they are not used, unless you tell it otherwise. I have no problem with Francois's suggestion for a seemless user interface, but await Arno's comments since he has implemented various multilisten servers before and is aware of the internal issues. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to several IP:Port ?
On Fri, Nov 26, 2010 at 4:28 PM, Angus Robertson - Magenta Systems Ltd an...@magsys.co.uk wrote: I do not see any need for multiple listening except 0.0.0.0 and ::0 as I have seen none of the servers doing it. No commercial server listens on 0.0.0.0 since commercial servers all have multiple IP addresses with multiple users and multiple applications, with the exception of IIS that hogs all addresses even if they are not used, unless you tell it otherwise. In IIS 7.5, there is the option of selecting All Unassigned. We have the 0.0.0.0 IPv4 equivalent of it in ICS but that does not cover IPv6 ::0. What I said is if we can provide a mechanism to cover really all unassigned, the rest could be managed with multiple server components. However if you want to have your own server listen an IP on the same port, you must set it a higher boot priority group. Otherwise forcing IIS to allocate even only one IP is impossible. SZ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to several IP:Port ?
Otherwise forcing IIS to allocate even only one IP is impossible. It is quite possible, my public server uses IIS/7 and an ICS web application on different IPs. There are NETSH HTTP IPLISTEN commands that force IIS to listen only on specific IPs, or just change some registry settings. My applications have arrays of server components to support multiple IPs and ports, all using the same events, but all created in code. Building multiple listens into the component is really only beneficial for those that prefer to drop one component on a form. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to severalIP:Port ?
Francois PIETTE wrote: This design would not break any existing code and would ease using the new multiple listen easy to implement in higher level components. It won't break existing code, but remains a hack IMO. All event assignments and unassignments have to be kept in sync. Also I would not bind the construction/destruction of the additional instances to the Addr string property (grrr), instead let the user explicitly create additional server instances. That could be, for instance, a method like: TWSocketServer.AddServer( const AServerClass: TWSocketServerClass; const AAddr: String; APort: String; AListenBackLog: Integer): TWSocketServer; virtual; .. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening toseveralIP:Port ?
This design would not break any existing code and would ease using the new multiple listen easy to implement in higher level components. It won't break existing code, but remains a hack IMO. I don't agree with you. Please explain. All event assignments and unassignments have to be kept in sync. They will be easily with setter and getter. Also I would not bind the construction/destruction of the additional instances to the Addr string property (grrr), Why grrr ? Please explain. instead let the user explicitly create additional server instances. That could be, for instance, a method like: TWSocketServer.AddServer( const AServerClass: TWSocketServerClass; const AAddr: String; APort: String; AListenBackLog: Integer): TWSocketServer; virtual; Such a method could complement the design I suggested, updating the addr and other properties. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to
Hello! Hello all, Since the Addr property is a string, the component could easily handle any special format in that field to support multiple listening IP:Port (Interface to listen to and port to use on that interface). I think something like /192.168.1.1:80/0.0.0.0:81/. IPv4 or IPv6 could be used. The component can easily find out that there is a multiple IP:Port because of the / at the beginning. The port property would be used as default value when no prot is specified in the Addr for a given IP. As an example: Addr := /192.168.1.1/192.168.2.1/10.1.2.3:80/ and Port := 81 would make listening on port 81 for 192.168.1.1 and 192.168.2.1, and listening on port 80 for interface 10.1.2.3. Now let's assume that application listens on two ip:port pairs, each for different kind of communication. For example, 192.168.1.1:2048 and 192.168.1.1:27015. Also, let's assume that user can define on which ip:port one of servers should listen (another ip:port pair would be fixed). What happens if user enters /192.168.1.1:2048/192.168.1.1:27015/ (for example, because he/she heard/read somewhere that with new ICS this is possible, and the application is based on ICS)? There would be clear conflict between either: - one of server components that now owns both ports and another that attempts to bind to owned port *or* - the second component that correctly owns 27015 port and first component that attempts to bind subservers to both 2048 and 27015 ports (depends on component creation order). While that change doesn't directly break existing code, it might introduce difficult to understand bugs, as seen above. One workaround I see now is to filter the user input, but that adds new code to equation. Another, and what I think the most reasonable, is to create a new class derived from TWSocketServer that would accept such input and leave original TWSocketServer untouched. Just my 5€. -- Piotr Dałek enigmati...@interia.pl -- Chcesz więcej zarabiać? ZmieƄ pracę na lepszą! http://linkint.pl/f2849 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: how to make it listening to
Since the Addr property is a string, the component could easily handle any special format in that field to support multiple listening IP:Port (Interface to listen to and port to use on that interface). I think something like /192.168.1.1:80/0.0.0.0:81/. IPv4 or IPv6 could be used. The component can easily find out that there is a multiple IP:Port because of the / at the beginning. The port property would be used as default value when no prot is specified in the Addr for a given IP. As an example: Addr := /192.168.1.1/192.168.2.1/10.1.2.3:80/ and Port := 81 would make listening on port 81 for 192.168.1.1 and 192.168.2.1, and listening on port 80 for interface 10.1.2.3. Now let's assume that application listens on two ip:port pairs, each for different kind of communication. For example, 192.168.1.1:2048 and 192.168.1.1:27015. Also, let's assume that user can define on which ip:port one of servers should listen (another ip:port pair would be fixed). What happens if user enters /192.168.1.1:2048/192.168.1.1:27015/ (for example, because he/she heard/read somewhere that with new ICS this is possible, and the application is based on ICS)? There would be clear conflict between either: - one of server components that now owns both ports and another that attempts to bind to owned port *or* - the second component that correctly owns 27015 port and first component that attempts to bind subservers to both 2048 and 27015 ports (depends on component creation order). This is not different than usual. Any application asking for to listen on a IP:port has to be ready to be refused because the port is already in use. Having multiple IP:port created by a single component doesn't change anything here. The application would handle it exactly the same way. While that change doesn't directly break existing code, it might introduce difficult to understand bugs, as seen above. I don't agree with you. The design as I suggest make it totally equivalent to having multiple independant TWSocketServer component with common event handlers. One workaround I see now is to filter the user input, but that adds new code to equation. Another, and what I think the most reasonable, is to create a new class derived from TWSocketServer that would accept such input and leave original TWSocketServer untouched. The design I suggest make TWSocketServer still behave exactly as currently, just allowing a new syntax for addr property. If not used, then no change in code is required. But yes, maybe a new derived from TWSocketServer could be used, or better maybe a new intermediate class between TWSocketServer and TCustomWSocketServer. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Dear damienwri...@segurosoft.com, I will be away on holiday from 27/7/2010 until 10/8/2010 and will be unable to deal with your recent message regarding `Re: [twsocket] TWSocketServer OnConnection event`. For technical support enquiries please email supp...@ietgroup.com or telephone 01442 878777. Best regards, Andrew Leiper IET Ltd 01442 878777 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Damien Wright wrote: I looked through the source and can see in the TriggerSessionAvailable handler the line If Error 0 then Exit; And this is done before the construction of a 'client socket' object with which to handle the connection and also prior to the point where the OnClientConnect method is called. So I am guessing an error is being passed in the LParam of the message. Hopefully by attaching the OnSessionAvailable event we might be able to capture what this error is and the be able to understand why this site has a particular problem. If that actually happens it's definitely not an application issue. According to the Microsoft docs there is just one possible error: WSAENETDOWN The network subsystem failed. http://msdn.microsoft.com/en-us/library/ms741540.aspx -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Dear arno.garr...@gmx.de, I will be away on holiday from 27/7/2010 until 10/8/2010 and will be unable to deal with your recent message regarding `Re: [twsocket] TWSocketServer OnConnection event`. For technical support enquiries please email supp...@ietgroup.com or telephone 01442 878777. Best regards, Andrew Leiper IET Ltd 01442 878777 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
I have been asked to investigate a strange issue we are encountering at a customer site in Mexico. I am a contractor for a company which supplied surveillance and monitoring software based on the ICS component set. The software runs fine on other sites with no problems encountered for over 8 months but on the site in Mexico after a matter of hours or days the software (and or server) crashes. The servers are all identical HP Blade servers running Windows Server 2003 vanilla installs. This is true of sites that are functioning and the ones in Mexico that are not. If the software runs fine on several indentical systems and fails on a single system, I would concentrate on what make that failing site different because it has to be different. Fist check the service pack level. I suggest first to verify that no malware is intercepting winsock calls. This is done by malware to capture trafic. Then, I would check if any suspect LSP is not installed on the system. Also check if some security products are not interfering with winsock: they frequently intercept winsock calls to block some kind of trafic. Those security products could be buggy. My analysis of the problem to date suggests that an OnClientConnect is firing but the passed Client object is incomplete or invalid. The code for the OnClientConnect event does not check the ErrorCode and accepts the connection but traffic appears not to flow correctly between client and server. I suggest checking the error code and reporting it into the logile for analisys. if I run NetStat on the server it appears a windows socket object is left in FIN-WAIT 1 or FIN-WAIT2 state. Eventually the system fails as all windows socket objects are expended and there is a catastrophic failure of the software and/or server. the steps that should be taken when an error does occur to ensure that the windows sockets are correctly 'cleaned up' and released back to the Operating System ? FIN-WAIT-1 and FIN-WAIT-2 means the orderly shutdown sequence is occuring but the remote site do not answer (Have a look here: http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm). An orderly shutdown is a multiple steps sequence between client and server. What is strange here is that FIN-WAIT-1 and FIN-WAIT-2 states are client side states, not server side. So it is possible that the socket you see in that sate are NOT the one failing. Maybe something else is failing (maybe in the same software) causing those sockets to be in those states and consume all available sockets which cause trouble in the software for accepting a new connection because accepting a new connection means creating a new socket. So I see the possibility that some other software or another part of your software has an issue with /client/ connection close, this result in a lot of sockets in the FIN-WAIT-1 or FIN-WAIT-2 state, consuming all available socket and making new connection acceptance failure. Why those client connexions could have problems with their server not answering ? This could be cause by malware sending forget IP packets to break existing connection or a misconfiguring security product (firewall) dropping packets, or simply an overloaded network segment which is dropping packets because trafic is too high. An overloaded layer 2 switch may simply drop packets when is it not able to switch the packets fast enough. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Dear francois.pie...@skynet.be, I will be away on holiday from 27/7/2010 until 10/8/2010 and will be unable to deal with your recent message regarding `Re: [twsocket] TWSocketServer OnConnection event`. For technical support enquiries please email supp...@ietgroup.com or telephone 01442 878777. Best regards, Andrew Leiper IET Ltd 01442 878777 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Interestingly the point you make about an overloaded network segment possibly causing dropped packets might be a likely candidate as the network is certainly more heavily utilised on that site than at any other and as we are not in control of any of the platform configurations, network schematics etc it is beyond our control/scope to do anything about. I am adding extra trace in 4 additional events and logging the socket states and error codes in the OnSessionAvailable, OnDebugAvailable, OnClientCreate and OnSessionClosed events. Previously we were not connecting any event handlers to these so were unable to log the state. Analysis by the Prime contractor on the customers site appeared to suggest that an FD_ACCEPT message was being processed but with an error code as they reported that using a network analysis trace the socket initialization was started correctly but that the log entries written in the OnClientConnect were not written. I looked through the source and can see in the TriggerSessionAvailable handler the line If Error 0 then Exit; And this is done before the construction of a 'client socket' object with which to handle the connection and also prior to the point where the OnClientConnect method is called. So I am guessing an error is being passed in the LParam of the message. Hopefully by attaching the OnSessionAvailable event we might be able to capture what this error is and the be able to understand why this site has a particular problem. If and when I receive these additional logs I will post any conclusions here. Best regards, Damien. -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Francois PIETTE Sent: Tuesday, August 03, 2010 5:19 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer OnConnection event I have been asked to investigate a strange issue we are encountering at a customer site in Mexico. I am a contractor for a company which supplied surveillance and monitoring software based on the ICS component set. The software runs fine on other sites with no problems encountered for over 8 months but on the site in Mexico after a matter of hours or days the software (and or server) crashes. The servers are all identical HP Blade servers running Windows Server 2003 vanilla installs. This is true of sites that are functioning and the ones in Mexico that are not. If the software runs fine on several indentical systems and fails on a single system, I would concentrate on what make that failing site different because it has to be different. Fist check the service pack level. I suggest first to verify that no malware is intercepting winsock calls. This is done by malware to capture trafic. Then, I would check if any suspect LSP is not installed on the system. Also check if some security products are not interfering with winsock: they frequently intercept winsock calls to block some kind of trafic. Those security products could be buggy. My analysis of the problem to date suggests that an OnClientConnect is firing but the passed Client object is incomplete or invalid. The code for the OnClientConnect event does not check the ErrorCode and accepts the connection but traffic appears not to flow correctly between client and server. I suggest checking the error code and reporting it into the logile for analisys. if I run NetStat on the server it appears a windows socket object is left in FIN-WAIT 1 or FIN-WAIT2 state. Eventually the system fails as all windows socket objects are expended and there is a catastrophic failure of the software and/or server. the steps that should be taken when an error does occur to ensure that the windows sockets are correctly 'cleaned up' and released back to the Operating System ? FIN-WAIT-1 and FIN-WAIT-2 means the orderly shutdown sequence is occuring but the remote site do not answer (Have a look here: http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm). An orderly shutdown is a multiple steps sequence between client and server. What is strange here is that FIN-WAIT-1 and FIN-WAIT-2 states are client side states, not server side. So it is possible that the socket you see in that sate are NOT the one failing. Maybe something else is failing (maybe in the same software) causing those sockets to be in those states and consume all available sockets which cause trouble in the software for accepting a new connection because accepting a new connection means creating a new socket. So I see the possibility that some other software or another part of your software has an issue with /client/ connection close, this result in a lot of sockets in the FIN-WAIT-1 or FIN-WAIT-2 state, consuming all available socket and making new connection acceptance failure. Why those client connexions could have problems with their server not answering ? This could be cause by malware sending forget IP packets to break existing
Re: [twsocket] TWSocketServer OnConnection event
Hello, Damien Wright wrote: and on each occasion if I run NetStat on the server it appears a windows socket object is left in FIN-WAIT 1 or FIN-WAIT2 state. Eventually the system fails as all windows socket objects are expended and there is a catastrophic failure of the software and/or server. Might be a FIN_WAIT DoS attack and the Mexico server hasn't got all security patches? http://www.recurity-labs.com/content/pub/Microsoft_Windows_CVE-2009-1926_MS09-048.txt http://www.microsoft.com/technet/security/bulletin/ms09-048.mspx -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Dear arno.garr...@gmx.de, I will be away on holiday from 27/7/2010 until 10/8/2010 and will be unable to deal with your recent message regarding `Re: [twsocket] TWSocketServer OnConnection event`. For technical support enquiries please email supp...@ietgroup.com or telephone 01442 878777. Best regards, Andrew Leiper IET Ltd 01442 878777 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer OnConnection event
Dear damienwri...@segurosoft.com, I will be away on holiday from 27/7/2010 until 10/8/2010 and will be unable to deal with your recent message regarding `[twsocket] TWSocketServer OnConnection event`. For technical support enquiries please email supp...@ietgroup.com or telephone 01442 878777. Best regards, Andrew Leiper IET Ltd 01442 878777 -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWsocketserver application - Broken link
I would like to know how to detect a client's broken link in an TWsocketserver application. Not a closed connection but really a broken one. An idle TCP connection does not send any data, so you will not know it's broken until it times out failing to send data. You can enable keep alive to send stuff regularly and force an error. You always get session closed, even for a clean close or failed open so you need to check the error number. Servers should also timeout idle connections, the latest wsocket has various timers for such purposes. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer: Clients disconnecting but ClientCountkeeps rising
robertoschler wrote: I have a server socket application that uses TWSocketServer. When clients disconnect, I do get see a SessionClosed event triggered for the client socket and then the TWSocketServer ClientDisconnect event handler fires. However, the ClientCount does not go down, and keeps incrementing with every new incoming connection, even though the Clients are disconnecting. I traced through the TWSocketServer code and I do see the RemoveList code fired for the component list object that is removing the disconnecting client socket (as expected), yet the ClientCount keeps rising. What could I be doing wrong that would prevent TWSocketServer from successfully decrementing the ClientCount? I've no idea, ClientCount returns Count of the client list so if the client is removed from the list ClientCount must be decremented. Are you able to reproduce this with some demo shipped with the ICS version you use? Which version do you use? -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
We need 5000+ clients stable work. Angus Robertson - Magenta Systems Ltd [EMAIL PROTECTED] wrote: How many client connections can produce as maximum TWSocketServer at the same time? (Every client will be sending ~2Kb every 3-5sec. Server will be rarely sending 1Kb broadcast to all clients.) Hundreds of connections, PC OS and memory dependent. I ran a test last week on XP with 250 clients running for seven days sending one line of text a second saved to 250 different files, the tasks were using about 5% CPU. You can test this yourself using my ComCap (captures text) and ComGen (generates text) applications that now have a limit of 500 clients. http://www.magsys.co.uk/comcap/cmbeta.asp Or programmatically with 'Magenta Systems IP Log Streaming Component' which is what ComCap and ComGen use, just create an array with 500 or more components in code: http://www.magsys.co.uk/delphi/magics.asp Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be - Never miss a thing. Make Yahoo your homepage. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
Hello Alexander, the only limit is the OS and available RAM. This is winsock issue not ICS. You need 10 KB non-paged RAM per socket. Non paged RAM can grow in OS until maximum 1/8 of fisical RAM. Note that other programs also can use non paged memory. that is each program that uses overlapped IO at least. For 5000+ connections I should advice a server OS. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Friday, March 28, 2008, 14:25, Alexander Makhanev wrote: We need 5000+ clients stable work. Angus Robertson - Magenta Systems Ltd [EMAIL PROTECTED] wrote: How many client connections can produce as maximum TWSocketServer at the same time? (Every client will be sending ~2Kb every 3-5sec. Server will be rarely sending 1Kb broadcast to all clients.) Hundreds of connections, PC OS and memory dependent. I ran a test last week on XP with 250 clients running for seven days sending one line of text a second saved to 250 different files, the tasks were using about 5% CPU. You can test this yourself using my ComCap (captures text) and ComGen (generates text) applications that now have a limit of 500 clients. http://www.magsys.co.uk/comcap/cmbeta.asp Or programmatically with 'Magenta Systems IP Log Streaming Component' which is what ComCap and ComGen use, just create an array with 500 or more components in code: http://www.magsys.co.uk/delphi/magics.asp Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be - Never miss a thing. Make Yahoo your homepage. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
We need 5000+ clients stable work. You need Windows /SERVER/ operating system. You need 4GB of RAM (even if there is free memory). You need to use ICS-V6. You need to carefully design you application because you'll reach many Windows limits (there is a 1 handles per process limit). Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
We have all of this (uses ics6, win server 2003, 2CPU XEON 3.2GHz, 4Gb RAM, 1Gb network). Seems like it's not an issue and there is something wrong with our server implementation (based on ics twsocketserver). I've asked for an advise on ics programmer who could help us in the neighbor message. Basicly our app is sending some streams (TMemoryStream) between server and clients (2Kb from clients every 3-5sec, rarely 1Kb broadcast from server to all clients). Francois Piette [EMAIL PROTECTED] wrote: We need 5000+ clients stable work. You need Windows /SERVER/ operating system. You need 4GB of RAM (even if there is free memory). You need to use ICS-V6. You need to carefully design you application because you'll reach many Windows limits (there is a 1 handles per process limit). Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be - Looking for last minute shopping deals? Find them fast with Yahoo! Search. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
We have all of this (uses ics6, win server 2003, 2CPU XEON 3.2GHz, 4Gb RAM, 1Gb network). Seems like it's not an issue and there is something wrong with our server implementation (based on ics twsocketserver). If you have a set-up that can easily generate 5,000 client sessions, it might be worth testing with Windows 2008, ideally 64-bit edition, which is now released and can be downloaded as a trial. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer maximum productivity
How many client connections can produce as maximum TWSocketServer at the same time? (Every client will be sending ~2Kb every 3-5sec. Server will be rarely sending 1Kb broadcast to all clients.) Hundreds of connections, PC OS and memory dependent. I ran a test last week on XP with 250 clients running for seven days sending one line of text a second saved to 250 different files, the tasks were using about 5% CPU. You can test this yourself using my ComCap (captures text) and ComGen (generates text) applications that now have a limit of 500 clients. http://www.magsys.co.uk/comcap/cmbeta.asp Or programmatically with 'Magenta Systems IP Log Streaming Component' which is what ComCap and ComGen use, just create an array with 500 or more components in code: http://www.magsys.co.uk/delphi/magics.asp Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
No worries! Here's the update on this: I have *slightly* modified my application based on the suggestions and insight I received from this list. When I say slightly I mean a lot, but that sounds too ominous :) First, I switched to TWSocketThrdServer without a hitch (hurray! for Arno's hard work on it). This introduced a new bottleneck: the database calls needed to be synchronized, and so it was basically the same as running on a single thread but with the additional overhead of thread creation and synchronization. The end result was that it made my server slower. Which brings me to the second change I made: I re-factored all database requests into a separate thread and completely de-coupled the other two working threads from having to perform any database access or additional management logic; they just post messages to the new thread with the data they need stored in the database. This introduce a whole new level of complexity: that of inter-thread communications and how to cope with un-received posted messages if the thread needs to abort unexpectedly (since the other threads now send and forget and expect the database thread to do its thing). And this led me to the final change in my application: The new database thread became the overall manager of the application: it governs all other threads, instructs them when to start and stop, and appropriately deals with anybody's impromptu demise. So now I have 3 worker threads: 1. Queue Manager: performs all database access, inter-thread management, application initialization and recovery, and overall management of the entire queue. 2. Queue Receiver: accepts incoming client connections and stores their data into the queue, then post the necessary information to the Manager, making sure state is always recoverable in case of failure. 3. Queue Dispatcher: scans periodically the queue and sends the messages via SMTP, then posts a message to the Manager announcing their result (whether success or failure) so that it can update the database record and remove the message from the queue. It also receives notifications from the Manager whenever new messages arrive of higher priority, so that it can interrupt its current scan and address those. Overall, the new design is more elegant and flexible, and still very stable; but more importantly, it is now considerably faster than before (orders of magnitude), and none of the connection issues that I was encountering are manifesting anymore. For the sake of comparisson, it now can take about 30 to 40 seconds to send 1000 messages to the Queue Server. And that's with a backlog of 50. A backlog of 5 takes a few seconds more because (at most) a handful of connections need to be retried (10061 error). A backlog of 10 succeeds without retries and takes roughly the same time. This means that it was my application design which was impeding the performance of TWSocketServer, and not an inherent issue with TWSocket itself (DOH!). System resources are limited, of course, so in my opinion our empirical analysis on the usage of the backlog is still valid: a larger number seems to affect performance negatively without any overall gain in availability, especially under heavy stress. In conclusion, as Arno and Wilfried suggested from the beginning (and as Francois has always claimed), TWSocket is fast, efficient and fully capable of handling thousands of concurrent connections, provided there are sufficient resources for it, and that no _extensive_processing_ is competing with the socket communication. How's that for an endorsement :) Thanks to all of you who offered help and suggestions. Cheers! -dZ. --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 12/5/2007 12:44:57 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog Hey DZ. Sorry, I didn't mean to drop out of this email thread. I have just been slammed for the last week and didn't have a chance to response to any of the further posts on this (they were buried in very long inbox). From what I see, Wilfried and Arno helped you out more than I would have anyway. Also, sorry I misunderstood your initial post about this. Story of my life... always coming in to the middle of a conversation confused and broke... ;) BTW, the pocket calculator comment was LOL... :) -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hey DZ. Sorry, I didn't mean to drop out of this email thread. I have just been slammed for the last week and didn't have a chance to response to any of the further posts on this (they were buried in very long inbox). From what I see, Wilfried and Arno helped you out more than I would have anyway. Also, sorry I misunderstood your initial post about this. Story of my life... always coming in to the middle of a conversation confused and broke... ;) BTW, the pocket calculator comment was LOL... :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of DZ-Jay Sent: Thursday, November 29, 2007 4:52 AM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer and backlog Wait, I'm sorry, I perhaps did not explain correctly: It was taking 5 to 7 minutes for the server to *process* the client's request to completion, not the connection. My tests, although quick and dirty, are intended to check the behaviour of my application as a whole, not just the connection. For the sake of understanding, here's a brief explanation of my project: Its an e-mail queue server; it has a listening thread running TWSocketServer, which will receive requests from my queue client. The client communicates with a custom line-based protocol, and sends the message data, which will then be stored in the queue (filesystem) by the listening thread. A separate thread periodically scans the queue directories and dispatches the messages to the SMTP server. The client builds and encodes the entire message on the fly, line by line as the data is sent, to avoid having the entire thing on memory at once. But that's not really important to this discussion (I'm just proud of it :). A large message may take a few seconds to transmit. My tests all send the same message: a multi-part message with alternative text and html parts, and a small (4Kb) binary attachment, encoded in MIME. The whole thing was about 14Kb (I think, I'm not sure). I was sending 1000 of these. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) He, he, he. If it was processing 1000 *connections* in 5 minutes, I'd say a pocket calculator! 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? Not the client, but the server is performing some integrity checks, file IO, and eventually storing a record of the transaction in the database. The client does indeed build the message on the fly, even encoding all content as lines are sent to the server (I'm sorry, there I go again, but I think this is pretty nifty :), but it doesn't start doing that until after the connection is established and the message data is going to be sent. Plus both the server and the client were running on the same development machine, along with Delphi-hog-my-memory-2006, in debug mode, with no optimizations. Moreover, the client test app has a TMemo, displaying the progress, and in my rush to make a quick and dirty test, the test app does not free the client objects (all 1000 of them), until it finishes. So the slowness wasn't unexpected. The point of my previous message was to show the difference between two tests, when the only variable was the backlog value: a backlog of 5 took less than half the time to do the exact same work as a backlog of 500. The problem that I see is that the TWSocketServer seems to be taking too long (relatively speaking) to accept the connections. My client seems to be able to send lots of connection requests before a single one is established, thus abusing and exceeding the backlog queue. Of course, it could be my application that is preventing TWSocketServer from doing its work effectively, and if so, then perhaps I should consider using a multi-threaded server. I cringe at that thought, though, because I had so many problems getting TWSocketThrdServer to run properly (due to my own lack of experience with this sort of thing). Any recommendations would be greatly appreciated. dZ. On Nov 28, 2007, at 18:47, Hoby Smith wrote: H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups
Re: [twsocket] TWSocketServer and backlog
On Nov 29, 2007, at 14:20, Arno Garrels wrote: Hard to tell, a good compromise is using TWSocketServer given any lengthy task is run in worker threads. I think separating socket IO work from other tasks by using worker threads for those tasks considered lengthy is the way to go. The definition of lengthy however is another story then. g The problem I have is that most of the processing is atomical to the client transaction, that is, -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Wait, I'm sorry, I perhaps did not explain correctly: It was taking 5 to 7 minutes for the server to *process* the client's request to completion, not the connection. My tests, although quick and dirty, are intended to check the behaviour of my application as a whole, not just the connection. For the sake of understanding, here's a brief explanation of my project: Its an e-mail queue server; it has a listening thread running TWSocketServer, which will receive requests from my queue client. The client communicates with a custom line-based protocol, and sends the message data, which will then be stored in the queue (filesystem) by the listening thread. A separate thread periodically scans the queue directories and dispatches the messages to the SMTP server. The client builds and encodes the entire message on the fly, line by line as the data is sent, to avoid having the entire thing on memory at once. But that's not really important to this discussion (I'm just proud of it :). A large message may take a few seconds to transmit. My tests all send the same message: a multi-part message with alternative text and html parts, and a small (4Kb) binary attachment, encoded in MIME. The whole thing was about 14Kb (I think, I'm not sure). I was sending 1000 of these. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) He, he, he. If it was processing 1000 *connections* in 5 minutes, I'd say a pocket calculator! 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? Not the client, but the server is performing some integrity checks, file IO, and eventually storing a record of the transaction in the database. The client does indeed build the message on the fly, even encoding all content as lines are sent to the server (I'm sorry, there I go again, but I think this is pretty nifty :), but it doesn't start doing that until after the connection is established and the message data is going to be sent. Plus both the server and the client were running on the same development machine, along with Delphi-hog-my-memory-2006, in debug mode, with no optimizations. Moreover, the client test app has a TMemo, displaying the progress, and in my rush to make a quick and dirty test, the test app does not free the client objects (all 1000 of them), until it finishes. So the slowness wasn't unexpected. The point of my previous message was to show the difference between two tests, when the only variable was the backlog value: a backlog of 5 took less than half the time to do the exact same work as a backlog of 500. The problem that I see is that the TWSocketServer seems to be taking too long (relatively speaking) to accept the connections. My client seems to be able to send lots of connection requests before a single one is established, thus abusing and exceeding the backlog queue. Of course, it could be my application that is preventing TWSocketServer from doing its work effectively, and if so, then perhaps I should consider using a multi-threaded server. I cringe at that thought, though, because I had so many problems getting TWSocketThrdServer to run properly (due to my own lack of experience with this sort of thing). Any recommendations would be greatly appreciated. dZ. On Nov 28, 2007, at 18:47, Hoby Smith wrote: H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? 3. Do you have any problems with other apps on your system running slow? Perhaps you have a bad driver or resource conflict with your NIC? Just some thoughts... Hoby -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
On Nov 29, 2007, at 06:10, Wilfried Mestdagh wrote: Hello DZ-Jay, So conclusion is that increasing the backlog does: - decrease the performance for accepting connections - decrease the overall performance of the application This seems to be the conclusion of mine and Huby's tests. Also: - connecting clients should have a range of retry's when refused, eventually with a random small delay. Agreed. For your application: Seems you have a lot of processing to do. While your code is executing no incoming socket can be accepting. Then maybe execute a certain block (the most time consuming one) in a separate thread ? You can exchange data between threads the most easy is by posting a message where a pointer to the data is in WParam argument. The pointer can be freed in the custom message handler. I will consider this. Thank you, Wilfried. However, the queue manager (listening) thread does not have a single large block of long-executing code, but very small blocks that each do a little work, which may be affecting performance: 1. It runs TWSocketServer, so it has to process all incoming connections and client communications. 2. For each client, it parses the incoming requests from the client to determine what needs to be done, and which state they are in. The request is a string in CMD:VALUE fomat. 3. If it's message data, it writes to the filesystem. 4. When done successfuly, it logs to the database. 5. When a message posted by the dispatcher thread announces that a message has been dispatched, the manager thread needs to log this to the database. And all throughout it is writing to a log file (at least while debugging), which needs to be synchronized among all threads. I will try to do some analysis to determine which portions are the bottleneck and see if they could be offset to a separate thread. I do not mind too much right now if the server runs a little slow; we can always re-factor it and optimize it in the future. But what I would like to avoid is rejecting connections too often because of a full backlog (which seems to be happening right now). Perhaps I should run the TWSocketServer on its own thread, and post messages from the clients to the queue manager thread to do the work? Although this seems too complex and expensive. It almost looks like each client should run on its own thread... :( dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
DZ-Jay wrote: On Nov 29, 2007, at 06:10, Wilfried Mestdagh wrote: Hello DZ-Jay, So conclusion is that increasing the backlog does: - decrease the performance for accepting connections - decrease the overall performance of the application This seems to be the conclusion of mine and Huby's tests. Strange, I never noticed something like that. You can exchange data between threads the most easy is by posting a message where a pointer to the data is in WParam argument. The pointer can be freed in the custom message handler. That's indeed the fastest way since the thread must not wait. Perhaps I should run the TWSocketServer on its own thread, and post messages from the clients to the queue manager thread to do the work? Although this seems too complex and expensive. It almost looks like each client should run on its own thread... :( I'm that sure: 1 - Stressing a server with 100 connection attempts per second is most likely not a real world scenario, except upon DoS attacks. 2 - Run your stress tester against IIS or other servers, I found that they were not able to accept more clients per second than my server. 3 - I played with different designs. a) Listening sockets in one thread, client sockets in another thread(s). This introduces a new problem, clients are accepted very fast, however the listening thread must synchronize with the client thread(s) which may take longer than with current TWSocketServer, I worked around by posting just the socket handle to the thread which was fast, however also rather complicated to handle all the client stuff/pool in the threads. b) Listening sockets in one thread, one thread per client. AFAIR without a thread pool accepting clients was slower than with TWSocketServer. c) I even hacked together a server that used M$ overlapped sockets, this was a rather disapointing discourse since performance was the same as with (a). The goal is to accept clients as fast as possible, once they are connected it won't hurt to let them wait some milliseconds. Before you rewrite your application I suggest you code some test apps. with different designs and compare their performance. -- Arno Garrels dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
--- Original Message --- From: Arno Garrels[mailto:[EMAIL PROTECTED] You can exchange data between threads the most easy is by posting a message where a pointer to the data is in WParam argument. The pointer can be freed in the custom message handler. That's indeed the fastest way since the thread must not wait. However, if the main thread notified the slave threads to quit, the last thread that quits may post messages (before receiving the WM_QUIT message) to the first one and fail, which will cause the memory in the message to not be freed (until the application finally quits). I don't know if this is a real concern, though. 1 - Stressing a server with 100 connection attempts per second is most likely not a real world scenario, except upon DoS attacks. I agree. However, this is very easily done by a brain-dead developer using my queue client class in a simple 'for' loop to send a lot of messages at once, say, an announcement to all our customers. I would like to prevent this as much as possible by improving connection acceptance speed on the server, or else I'll have to cripple the client somehow. Do not underestimate the tenacity of morons. :) 2 - Run your stress tester against IIS or other servers, I found that they were not able to accept more clients per second than my server. I'm sure this is true. I am able to avoid the whole issue by responsibly designing the client application: send the next connection request after the first one triggers OnSessionConnected, or connecting only a few clients at a time, then pause until they are done. This not only improves performance of the server, but it prevents an inadvertent DoS attack from an application that needs to send lots of messages at once. 3 - I played with different designs. Which would you consider to work best? The goal is to accept clients as fast as possible, once they are connected it won't hurt to let them wait some milliseconds. This is indeed my goal. Would it make sense to have a pool of listening sockets in a separate (single) thread that will post a message to the (single) working thread with the socket handle? That way the connections can be established quickly, and my server can continue doing its processing within a single thread so that I don't have to redesign it right now. -dZ. Sent: 11/29/2007 1:52:38 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog DZ-Jay wrote: On Nov 29, 2007, at 06:10, Wilfried Mestdagh wrote: Hello DZ-Jay, So conclusion is that increasing the backlog does: - decrease the performance for accepting connections - decrease the overall performance of the application This seems to be the conclusion of mine and Huby's tests. Strange, I never noticed something like that. Perhaps I should run the TWSocketServer on its own thread, and post messages from the clients to the queue manager thread to do the work? Although this seems too complex and expensive. It almost looks like each client should run on its own thread... :( I'm that sure: 1 - Stressing a server with 100 connection attempts per second is most likely not a real world scenario, except upon DoS attacks. 2 - Run your stress tester against IIS or other servers, I found that they were not able to accept more clients per second than my server. 3 - I played with different designs. a) Listening sockets in one thread, client sockets in another thread(s). This introduces a new problem, clients are accepted very fast, however the listening thread must synchronize with the client thread(s) which may take longer than with current TWSocketServer, I worked around by posting just the socket handle to the thread which was fast, however also rather complicated to handle all the client stuff/pool in the threads. b) Listening sockets in one thread, one thread per client. AFAIR without a thread pool accepting clients was slower than with TWSocketServer. c) I even hacked together a server that used M$ overlapped sockets, this was a rather disapointing discourse since performance was the same as with (a). The goal is to accept clients as fast as possible, once they are connected it won't hurt to let them wait some milliseconds. Before you rewrite your application I suggest you code some test apps. with different designs and compare their performance. -- Arno Garrels dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
[EMAIL PROTECTED] wrote: --- Original Message --- From: Arno Garrels[mailto:[EMAIL PROTECTED] You can exchange data between threads the most easy is by posting a message where a pointer to the data is in WParam argument. The pointer can be freed in the custom message handler. That's indeed the fastest way since the thread must not wait. However, if the main thread notified the slave threads to quit, the last thread that quits may post messages (before receiving the WM_QUIT message) to the first one and fail, which will cause the memory in the message to not be freed (until the application finally quits). I don't know if this is a real concern, though. When the process died the memory is given back to the OS anyway, so no problem. PostMessage() will return False on failure, in this case the memory can be released by the calling thread. 1 - Stressing a server with 100 connection attempts per second is most likely not a real world scenario, except upon DoS attacks. I agree. However, It's easy to DoS any server ;-) 2 - Run your stress tester against IIS or other servers, I found that they were not able to accept more clients per second than my server. I'm sure this is true. 3 - I played with different designs. Which would you consider to work best? Hard to tell, a good compromise is using TWSocketServer given any lengthy task is run in worker threads. I think separating socket IO work from other tasks by using worker threads for those tasks considered lengthy is the way to go. The definition of lengthy however is another story then. g -- Arno Garrels -dZ. Sent: 11/29/2007 1:52:38 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog DZ-Jay wrote: On Nov 29, 2007, at 06:10, Wilfried Mestdagh wrote: Hello DZ-Jay, So conclusion is that increasing the backlog does: - decrease the performance for accepting connections - decrease the overall performance of the application This seems to be the conclusion of mine and Huby's tests. Strange, I never noticed something like that. Perhaps I should run the TWSocketServer on its own thread, and post messages from the clients to the queue manager thread to do the work? Although this seems too complex and expensive. It almost looks like each client should run on its own thread... :( I'm that sure: 1 - Stressing a server with 100 connection attempts per second is most likely not a real world scenario, except upon DoS attacks. 2 - Run your stress tester against IIS or other servers, I found that they were not able to accept more clients per second than my server. 3 - I played with different designs. a) Listening sockets in one thread, client sockets in another thread(s). This introduces a new problem, clients are accepted very fast, however the listening thread must synchronize with the client thread(s) which may take longer than with current TWSocketServer, I worked around by posting just the socket handle to the thread which was fast, however also rather complicated to handle all the client stuff/pool in the threads. b) Listening sockets in one thread, one thread per client. AFAIR without a thread pool accepting clients was slower than with TWSocketServer. c) I even hacked together a server that used M$ overlapped sockets, this was a rather disapointing discourse since performance was the same as with (a). The goal is to accept clients as fast as possible, once they are connected it won't hurt to let them wait some milliseconds. Before you rewrite your application I suggest you code some test apps. with different designs and compare their performance. -- Arno Garrels dZ. -- DZ-Jay [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
I always use 500, no problems yet Paul - Original Message - From: [EMAIL PROTECTED] To: twsocket@elists.org Sent: Wednesday, November 28, 2007 6:27 PM Subject: [twsocket] TWSocketServer and backlog Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
--- Original Message --- From: Paul[mailto:[EMAIL PROTECTED] I always use 500, no problems yet Thanks for the quick reply. Then, is there a particular reason why it defaults to 5? It seems too low for all but the most trivial applications (given that spawning the client object and dupping the socket seems to take a relatively long time). -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Paul wrote: I always use 500, no problems yet But the ListenbacklogQueue is limited in size depending on the OS (cannot recall the values, however it's far less then 500, AFAIR). The more blocking the server behaves the earlier you get 10061 back from a connect. Simple test is with TcpSrv Demo, with logging to the memo enabled I get the first error 10061 after 100-200 connects (10ms intervals). Turning off logging to memo establishes several thousand connections without any error easily. -- Arno Garrels Paul - Original Message - From: [EMAIL PROTECTED] To: twsocket@elists.org Sent: Wednesday, November 28, 2007 6:27 PM Subject: [twsocket] TWSocketServer and backlog Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hello dz, a client application should do at least a few (or infinity) retry's if connection fails. so normally not needed to increase it. On the other hand it does no harm to increase it. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Wednesday, November 28, 2007, 18:27, [EMAIL PROTECTED] wrote: Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hello dz, I think 5 is the winsock default value --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Wednesday, November 28, 2007, 19:01, [EMAIL PROTECTED] wrote: --- Original Message --- From: Paul[mailto:[EMAIL PROTECTED] I always use 500, no problems yet Thanks for the quick reply. Then, is there a particular reason why it defaults to 5? It seems too low for all but the most trivial applications (given that spawning the client object and dupping the socket seems to take a relatively long time). -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hello: The problem with retrying is that it is not the same as a server full error when the maximum number of clients is reached; 100061 is essentially a port not open error, which is the same error you would get if the server is not running. So there is no real way to know that the listener is currently busy and the backlog full, or if the server is listening on a different port or disabled completely. I will certainly increase the backlog on my server, but will also consider building a number of retries in the connection routine of the client class. Thanks for the help. -dZ. --- Original Message --- From: Wilfried Mestdagh[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 2:26:49 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog Hello dz, a client application should do at least a few (or infinity) retry's if connection fails. so normally not needed to increase it. On the other hand it does no harm to increase it. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Wednesday, November 28, 2007, 18:27, [EMAIL PROTECTED] wrote: Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
FYI... I ran into an issue with some test code I wrote a few months ago, which related to the backlog setting, as well as the annoying issue with Winsock running out of local ports. In my test, I was attempting to see how many connections could be handled by a particular process over a period of time. I believe my results showed that increasing this value can have a very negative effect on performance. Basically, the issue is inherent in how the TCP stack is implemented, not in how a particular app services the stack. I found that surpassing a particular connection rate threshold would result in an exponential gain in processing time on the listening stack. Meaning, the TCP stack performance decreases dramatically as you increase the number of pending connections, when the listening socket is receiving a high rate of connection requests. My assumption is that this is due to the increased overhead in managing the backlog queue. Given this, I made two observations, which may be wrong, but made sense to me. First, this is why the Winsock default is 5. I imagine that the Winsock stack implementation was designed with the perspective that if the backlog is actually filling up enough to reach 5 or more, then something is wrong. Probably, a couple more might be ok, but my results showed that as you increased this value under heavy load, your connection rate was very unpredictable, as well as instable (lots of failed connects). For the TCP/IP stack to be effective, it must be responsive enough to handle the low level connection requests in a timely fashion. If not, then you have a major low level servicing problem or the machine is seriously overloaded with TCP requests. In which case, you want to get connection errors, rather than an overloaded backlog scenario. Second, increasing this value surely creates a greater DOS attack surface, making you more vulnerable to bursts of socket open requests, and surely would make the effects of such an attack even worse. This might also be why the Winsock default is 5. However, as I personally don't think that there is really a practical solution to a well designed DOS attack, then this might not really be relevant. Nonetheless, it might be something you need to consider. So, given that, I personally don't recommend increasing the value. If your app can't service the stack with a backlog setting close to 5, then your system is just overloaded or not responsive for some reason. Anyway, that is what I determined from my testing results. If anyone has found to the contrary, please feel free to correct me... :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 12:58 PM To: twsocket@elists.org Subject: Re: [twsocket] TWSocketServer and backlog Hello: The problem with retrying is that it is not the same as a server full error when the maximum number of clients is reached; 100061 is essentially a port not open error, which is the same error you would get if the server is not running. So there is no real way to know that the listener is currently busy and the backlog full, or if the server is listening on a different port or disabled completely. I will certainly increase the backlog on my server, but will also consider building a number of retries in the connection routine of the client class. Thanks for the help. -dZ. --- Original Message --- From: Wilfried Mestdagh[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 2:26:49 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog Hello dz, a client application should do at least a few (or infinity) retry's if connection fails. so normally not needed to increase it. On the other hand it does no harm to increase it. --- Rgds, Wilfried [TeamICS] http://www.overbyte.be/eng/overbyte/teamics.html http://www.mestdagh.biz Wednesday, November 28, 2007, 18:27, [EMAIL PROTECTED] wrote: Hello: While stress-testing my application, I noticed that I am able to send substantially many more connections in the time it takes the TWSocketServer to handle the incomming requests, causing the default backlog to fill up quickly. Obviously, I can increase the number, but seeing that the default is 5 (which seems rather low to me), I'm thinking that perhaps there may be a concern in setting this too high. Does anybody know what I should take into consideration before changing this value, and if there are any concerns with it being too high? Thanks, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
Hello: Thank you for your very informative response. I was performing some tests on my server application by continually increasing the backlog value with some mixed results, which seem to coincide with your empirical analysis. I kept increasing the backlog value up until I reached 1000, but to my surprise, I noticed that the connections started failing after about 230 requests, out of 1000 clients. These were the first 230 requests, so the backlog queue was significantly less than its maximum. I also thought I noticed that the server was taking longer to respond, but didn't think much of it at the time. However, after reading your post I decided to try once again with a backlog of 5, and set a retry loop every time a connection failed. As expected, the connections started failing almost immediately after the test started. But much to my surprise, the connections were handled quicker -- sometimes orders of magnitude faster than before! As a reference, using my localhost as the server and client, with a test application spawning 1000 clients to connect one right after the other, and re-trying if they failed, it took about 5 to 7 minutes to process the entire lot; while it only took about 2 minutes to process with a backlog of 5. The test with a backlog limit of 5 retried much more times, of course, but when connections were established, they were processed faster. Still, it seems to me that TWSocketServer is taking too long to process incoming connections, as many connections can be queued in the backlog while its instantiating the client and dupping the socket. Any thoughts on this? -dZ. --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 5:31:09 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog FYI... I ran into an issue with some test code I wrote a few months ago, which related to the backlog setting, as well as the annoying issue with Winsock running out of local ports. In my test, I was attempting to see how many connections could be handled by a particular process over a period of time. I believe my results showed that increasing this value can have a very negative effect on performance. Basically, the issue is inherent in how the TCP stack is implemented, not in how a particular app services the stack. I found that surpassing a particular connection rate threshold would result in an exponential gain in processing time on the listening stack. Meaning, the TCP stack performance decreases dramatically as you increase the number of pending connections, when the listening socket is receiving a high rate of connection requests. My assumption is that this is due to the increased overhead in managing the backlog queue. Given this, I made two observations, which may be wrong, but made sense to me. First, this is why the Winsock default is 5. I imagine that the Winsock stack implementation was designed with the perspective that if the backlog is actually filling up enough to reach 5 or more, then something is wrong. Probably, a couple more might be ok, but my results showed that as you increased this value under heavy load, your connection rate was very unpredictable, as well as instable (lots of failed connects). For the TCP/IP stack to be effective, it must be responsive enough to handle the low level connection requests in a timely fashion. If not, then you have a major low level servicing problem or the machine is seriously overloaded with TCP requests. In which case, you want to get connection errors, rather than an overloaded backlog scenario. Second, increasing this value surely creates a greater DOS attack surface, making you more vulnerable to bursts of socket open requests, and surely would make the effects of such an attack even worse. This might also be why the Winsock default is 5. However, as I personally don't think that there is really a practical solution to a well designed DOS attack, then this might not really be relevant. Nonetheless, it might be something you need to consider. So, given that, I personally don't recommend increasing the value. If your app can't service the stack with a backlog setting close to 5, then your system is just overloaded or not responsive for some reason. Anyway, that is what I determined from my testing results. If anyone has found to the contrary, please feel free to correct me... :) -Original Message- -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer and backlog
H... If it is taking your system 5 to 7 MINUTES to process 1000 connect / disconnect cycles, then something is very wrong. I would have to rerun my tests, but I am thinking that I was doing 1K connect / disconnects in about 10 to 15 seconds when running both server and client on a single core P4. Perhaps a little faster using several client instances at the same time, although the performance maxed quickly on a single core CPU. I believe it was much faster on a 4 way Xeon machine I tested it on. I can get more specific stats for you, if you want them. But, whatever my specific results were, 5 to 7 MINUTES is just WAY off. 1. What kind of machine is it? Commodore 64? TS-1000? TRS-80? Just kidding... ;) 2. Is your client class on the server initiating a bunch of additional processes, like database lookups or something? 3. Do you have any problems with other apps on your system running slow? Perhaps you have a bad driver or resource conflict with your NIC? Just some thoughts... Hoby -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 28, 2007 4:30 PM To: twsocket@elists.org Subject: Re: [twsocket] TWSocketServer and backlog Hello: Thank you for your very informative response. I was performing some tests on my server application by continually increasing the backlog value with some mixed results, which seem to coincide with your empirical analysis. I kept increasing the backlog value up until I reached 1000, but to my surprise, I noticed that the connections started failing after about 230 requests, out of 1000 clients. These were the first 230 requests, so the backlog queue was significantly less than its maximum. I also thought I noticed that the server was taking longer to respond, but didn't think much of it at the time. However, after reading your post I decided to try once again with a backlog of 5, and set a retry loop every time a connection failed. As expected, the connections started failing almost immediately after the test started. But much to my surprise, the connections were handled quicker -- sometimes orders of magnitude faster than before! As a reference, using my localhost as the server and client, with a test application spawning 1000 clients to connect one right after the other, and re-trying if they failed, it took about 5 to 7 minutes to process the entire lot; while it only took about 2 minutes to process with a backlog of 5. The test with a backlog limit of 5 retried much more times, of course, but when connections were established, they were processed faster. Still, it seems to me that TWSocketServer is taking too long to process incoming connections, as many connections can be queued in the backlog while its instantiating the client and dupping the socket. Any thoughts on this? -dZ. --- Original Message --- From: Hoby Smith[mailto:[EMAIL PROTECTED] Sent: 11/28/2007 5:31:09 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer and backlog FYI... I ran into an issue with some test code I wrote a few months ago, which related to the backlog setting, as well as the annoying issue with Winsock running out of local ports. In my test, I was attempting to see how many connections could be handled by a particular process over a period of time. I believe my results showed that increasing this value can have a very negative effect on performance. Basically, the issue is inherent in how the TCP stack is implemented, not in how a particular app services the stack. I found that surpassing a particular connection rate threshold would result in an exponential gain in processing time on the listening stack. Meaning, the TCP stack performance decreases dramatically as you increase the number of pending connections, when the listening socket is receiving a high rate of connection requests. My assumption is that this is due to the increased overhead in managing the backlog queue. Given this, I made two observations, which may be wrong, but made sense to me. First, this is why the Winsock default is 5. I imagine that the Winsock stack implementation was designed with the perspective that if the backlog is actually filling up enough to reach 5 or more, then something is wrong. Probably, a couple more might be ok, but my results showed that as you increased this value under heavy load, your connection rate was very unpredictable, as well as instable (lots of failed connects). For the TCP/IP stack to be effective, it must be responsive enough to handle the low level connection requests in a timely fashion. If not, then you have a major low level servicing problem or the machine is seriously overloaded with TCP requests. In which case, you want to get connection errors, rather than an overloaded backlog scenario. Second, increasing this value surely creates a greater DOS attack surface, making you more
Re: [twsocket] TWSocketServer Client.Shutdown(1) does not destroy it
I believe I solved it: It turns out that my application was intercepting the WM_CLIENT_CLOSED message mistakenly. My application use TWSocketServer within a worker thread, and this worker thread has a custom message dispatcher so that it can process messages sent to the thread without a hidden window. The problem was that I forgot to check if the Msg.HWND property was 0 before calling the custom dispatcher. Since I solved my own problem, and it has nothing to do with TWSocket, this message is not Off Topic. -dZ. --- Original Message --- From: [EMAIL PROTECTED]:[EMAIL PROTECTED] Sent: 10/17/2007 2:36:00 PM To : twsocket@elists.org Cc : Subject : RE: Re: [twsocket] TWSocketServer Client.Shutdown(1) does not destroy it Hello: Its worse... I also noticed that TWSocketClient.TriggerSessionClosed() is triggered by the client, but the message posted is never received by the server. This happens even then the client drops the connection. So, for some reason, the clients are never destroyed. I'm sure that its something to do with my code, but can anybody offer some suggestions as to what may cause this behaviour? Thanks, -dZ. --- Original Message --- From: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Sent: 10/17/2007 2:06:48 PM To : twsocket@elists.org Cc : Subject : RE: [twsocket] TWSocketServer Client.Shutdown(1) does not destroy it Hello: I just started noticing this behaviour today, and I seem to recall it working differently: When handling the OnClientDataAvailable event, if I determine that the client needs to be disconnected, I call (Sender As TMyClient).Shutdown(1). The connection is dropped fine, but the client object is never freed until the server shuts down completely. Here's a sample of my code: Procedure TMyServer.ClientDataAvailable(Sender: TObject; Error: Word); Begin With (Sender As TMyClient) Do Begin If (SomethingBadHappened) Then Begin SendLine('Error!'); Shutdown(1); End; End; End; I call Shutdown(1) so that the connection is dropped gracefully and the error response is received by the client. I don't recall changing this recently, so I'm confused as to why it would have been destroying the client before and not now (I may have changed some of the default properties, though I don't recall anything pertinent to this issue). Perhaps there's an even better way? Thanks for the help, -dZ. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
Hello Engi, Does thread C have a message pump? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 1:09 PM Subject: [twsocket] TWSocketServer in thread Hi, My program works with the main form and 3 threads (A, B C). Two of this threads (A B) are created by main form and the third (C) is created by thread A. Each thread create, run and destruct a server (TWSocketServer) in Execute() method. For thread A and B, my client application tcp/ip connections are ok. But the server in thread C don't answer to requests. My client can connect to it, that's all. Even the banner is not sent. Any suggestions will be appreciated. Thanks, Engi -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
I heard spoken several times of 'message pump' but I don't really understand what that means. So, I don't think there is a message pump in my thread. Could you explain me this concept ? My Execute() method is composed with a while(!terminated) loop which does the job. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 12:12 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Hello Engi, Does thread C have a message pump? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 1:09 PM Subject: [twsocket] TWSocketServer in thread Hi, My program works with the main form and 3 threads (A, B C). Two of this threads (A B) are created by main form and the third (C) is created by thread A. Each thread create, run and destruct a server (TWSocketServer) in Execute() method. For thread A and B, my client application tcp/ip connections are ok. But the server in thread C don't answer to requests. My client can connect to it, that's all. Even the banner is not sent. Any suggestions will be appreciated. Thanks, Engi -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
Do you have GetMessage or PeekMessage called in a loop in while(!Terminated)? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 2:32 PM Subject: Re: [twsocket] TWSocketServer in thread I heard spoken several times of 'message pump' but I don't really understand what that means. So, I don't think there is a message pump in my thread. Could you explain me this concept ? My Execute() method is composed with a while(!terminated) loop which does the job. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 12:12 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Hello Engi, Does thread C have a message pump? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 1:09 PM Subject: [twsocket] TWSocketServer in thread Hi, My program works with the main form and 3 threads (A, B C). Two of this threads (A B) are created by main form and the third (C) is created by thread A. Each thread create, run and destruct a server (TWSocketServer) in Execute() method. For thread A and B, my client application tcp/ip connections are ok. But the server in thread C don't answer to requests. My client can connect to it, that's all. Even the banner is not sent. Any suggestions will be appreciated. Thanks, Engi -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
No -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 2:46 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Do you have GetMessage or PeekMessage called in a loop in while(!Terminated)? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 2:32 PM Subject: Re: [twsocket] TWSocketServer in thread I heard spoken several times of 'message pump' but I don't really understand what that means. So, I don't think there is a message pump in my thread. Could you explain me this concept ? My Execute() method is composed with a while(!terminated) loop which does the job. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 12:12 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Hello Engi, Does thread C have a message pump? Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 1:09 PM Subject: [twsocket] TWSocketServer in thread Hi, My program works with the main form and 3 threads (A, B C). Two of this threads (A B) are created by main form and the third (C) is created by thread A. Each thread create, run and destruct a server (TWSocketServer) in Execute() method. For thread A and B, my client application tcp/ip connections are ok. But the server in thread C don't answer to requests. My client can connect to it, that's all. Even the banner is not sent. Any suggestions will be appreciated. Thanks, Engi -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be __ NOD32 2363 (20070629) Information __ This message was checked by NOD32 antivirus system. http://www.eset.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
Ok. Then take a look at WSocket.MessagePump(). You need to call something like that in a loop. When WM_QUIT is issued GetMessage returns false and exits. Otherwise it returns true. Here is my message pump: void __fastcall ReverseProxyClientThread::Execute() { // Place thread code here while(!Terminated) messagePump(); } //--- void __fastcall ReverseProxyClientThread::messagePump(void) { MSG msg; while(GetMessage(msg, NULL, 0, 0) !Terminated) { DispatchMessage(msg); } //--- Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 3:50 PM Subject: Re: [twsocket] TWSocketServer in thread No -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 2:46 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Do you have GetMessage or PeekMessage called in a loop in while(!Terminated)? Regards, SZ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
Here is a part of my Execute() code : try { while (GetMessage(msg, NULL, 0, 0) !Terminated) { DispatchMessage(msg); if (!DialogError) { ... ... ... } Sleep(1); } } catch(Exception e) { if (DebugActif) OutputDebugString(*** exception: + e.Message); } Is it correct ? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 2:56 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Ok. Then take a look at WSocket.MessagePump(). You need to call something like that in a loop. When WM_QUIT is issued GetMessage returns false and exits. Otherwise it returns true. Here is my message pump: void __fastcall ReverseProxyClientThread::Execute() { // Place thread code here while(!Terminated) messagePump(); } // --- void __fastcall ReverseProxyClientThread::messagePump(void) { MSG msg; while(GetMessage(msg, NULL, 0, 0) !Terminated) { DispatchMessage(msg); } // --- Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 3:50 PM Subject: Re: [twsocket] TWSocketServer in thread No -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 2:46 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Do you have GetMessage or PeekMessage called in a loop in while(!Terminated)? Regards, SZ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be __ NOD32 2363 (20070629) Information __ This message was checked by NOD32 antivirus system. http://www.eset.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] TWSocketServer in thread
What are the diffs between this functions ? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fastream Technologies Sent: Friday, June 29, 2007 3:28 PM To: ICS support mailing Subject: Re: [twsocket] TWSocketServer in thread Hello, Instead of Sleep(), why don't you Pause() the twsocket? That would be better IMHO. Regards, SZ - Original Message - From: Engi [EMAIL PROTECTED] To: 'ICS support mailing' twsocket@elists.org Sent: Friday, June 29, 2007 4:16 PM Subject: Re: [twsocket] TWSocketServer in thread Here is a part of my Execute() code : try { while (GetMessage(msg, NULL, 0, 0) !Terminated) { DispatchMessage(msg); if (!DialogError) { ... ... ... } Sleep(1); } } catch(Exception e) { if (DebugActif) OutputDebugString(*** exception: + e.Message); } Is it correct ? -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be __ NOD32 2363 (20070629) Information __ This message was checked by NOD32 antivirus system. http://www.eset.com -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be