Re: 2 uni-directional TCP connection good?
On Fri, 20 Mar 2009 01:56:56 -0700 Michael David Crawford m...@prgmr.com wrote: Yoshihiro Ota wrote: I saw a program that opens 2 TCP connections. One connection is only used for server to client messaging only and the other connection is used only for client to server messaging. 2. He also said that it would also waste network bandwidth. You have a two-way communication no matter what you do. But if you don't actually use inbound direction, all it gets used for is the receipt of ACK packets. That is, the inbound connection is used to make the data transfer reliable. If you don't have any payload data on the inbound connection, then the outbound connection won't have any ACK packets. If you're sending payload data, the ACK info can hitchhike along with the payload packets, thus saving bandwidth. But if you're not sending any payload data at all, there will be packets transmitted which contain the ACKs and nothing else. The extra network overhead will be modest if you're sending a lot of data all at once, say transferring a large file. But if very little data is sent per packet, say individual characters in a telnet connection, the overhead would be very high. So far until this, this was what I had though and learned about TCP connections. If you have a single connection with payload data in both directions, then the ACKs will almost always ride along with some payload data. The only time a packet will contain nothing but an ACK will be when some data was transmitted, but none is to be received at the time. Mike However, I had forgotten this case. This can explain he even said that using 2 TCP connection would cut the bandwidth into half. This sounds like the case he was referring to. Thanks, Hiro ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good
On Mon, 23 Mar 2009 08:20:20 + (GMT) Robert Watson rwat...@freebsd.org wrote: On Sun, 22 Mar 2009, Yoshihiro Ota wrote: On Fri, 20 Mar 2009, Yoshihiro Ota wrote: 1. With TCP connections, only sender side can detect some communication issues passively if happened. By using two connections, you lost that ability by your self. I agree on this one. Could you expand a bit on this point? While the connection creation process (usually) asymmetric, once the connection is built it's essentially the same state machine on both sides of the connection, and socket semantics with respect to the state machine are effectively identical. Application on both sides should be able to detect disconnect, monitor connection state using TCP_INFO, etc. What I meant was that there were cases when a receiver could not tell weather no data was coming or communication was interrupted. Once connection is established, a route is available between a server and a client. Let's say this route is broken for some reasons, i.e. someone unplugged a cable or a firewall started dropping or rejecting between these server and client, a sender may not notice as soon as it happens but at least, a sender knows a massages was not delivered right. On the other hand, receiver side does not have any idea that a message delivery failure has happened at all or for a while unless using heartbeat messages in upper layer. KEEP_ALIVE option seems to be implementation dependent such that you cannot assure TCP connection availability for every minute. This is generally considered a robustness property rather than a fragility issue, but yes: if you need a liveliness property for idle connections with TCP, it's something you have to implement at the application layer, and many protocols indeed do this. I don't see that this is an argument for using two TCP connections as opposed to one, however. If you're interested in alternative protocols, however, SCTP allows a number of these protocol behaviors to be modified, and includes support for a heartbeat. Actually, the programs I had problems with were ggated/ggatec. As I look back my records, I found a problem with ggate when I had a slow ggated server (Celeron 400MHz) and a fast ggatec client (AMD Turion(tm) 64 X2 1.9GHz). While the client was writing a large file to the server, there were often major delays in its communication. While this was happening, I observed with systat -vm and saw interrupts on the NIC was 1 per a couple seconds. Indeed, I tested SCTP with ggate but as far as I remember, STCP seemed to have problems such that ggate couldn't establish a connection back in 7.0-RELEASE. I tested the same code in 7.1-RELEASE just before and it appeared to be working. It was just an experiment back in a year ago and couldn't find a reason easily. So, I gave up. Recently, ggate became necessity. I enhanced some of debug capabilities and found that when this happens, sequence numbers in ggate are out of sync. For example, while this is happening, ggatec reports it has finished send() with seq# 186, but at the same exact time, ggated reports it has only recv()ed up to seq# 184. They get synced eventually, but it takes long time and happens frequently. Meanwhile, terminal does not response in timely manner. This hiccup doesn't happen if client is doing read-access to ggated. FTP or nfs doesn't have similar hiccup and communicate at its max speed, 100-Base T. Both ggatec and ggated creates 2 TCP connections and 2 threads for blocking reads and writes. When I mentioned this problem to the friend mentioned above, he suggested to change 2 uni-directional TCP connections to 1 bi-directional one. In fact, this fixed the problem. At the same time, he also expressed that 2 uni-directional TCP connection was so bad no one should not use like it must be prohibited at all. So, I wondered and wanted to know how other people would say about it. If using 2 uni-directional TCP communication is not that bad idea or something prohibited, then we may have some issues in TCP layer. If anyone is interested in looking into, I can elaborate other observations as well. Thanks, Hiro ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good?
What I meant was that there were cases when a receiver could not tell weather no data was coming or communication was interrupted. Once connection is established, a route is available between a server and a client. Let's say this route is broken for some reasons, i.e. someone unplugged a cable or a firewall started dropping or rejecting between these server and client, a sender may not notice as soon as it happens but at least, a sender knows a massages was not delivered right. On the other hand, receiver side does not have any idea that a message delivery failure has happened at all or for a while unless using heartbeat messages in upper layer. KEEP_ALIVE option seems to be implementation dependent such that you cannot assure TCP connection availability for every minute. The whole point of TCP (vs IP alone, or UDP) is to establish reliable end-to-end communication over unreliable underlying links. If a packet is corrupted or lost, it gets resent. If a route goes down, and an alternate is available, TCP will -- eventually -- find it and recover. If the last (or only) route goes down, TCP will in principle wait indefinitely for a route to become available, whether by reestablishment of the original or provision of an alternative. So you are correct that a receiver can't tell the difference between a loss of connectivity and the sender having crashed, however the situation is entirely symmetric: the sender can't tell the difference either. It all gets sorted out when communication is reestablished; at that point traffic will resume (if the link had been down) or the uncrashed end will get a connection reset (if its peer had crashed). The practice of sending keep-alive packets simply converts a temporary (thus potentially recoverable) communication loss into what amounts to an unrecoverable crash of whichever end gets impatient first. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good
On Sun, 22 Mar 2009, Yoshihiro Ota wrote: On Fri, 20 Mar 2009, Yoshihiro Ota wrote: 1. With TCP connections, only sender side can detect some communication issues passively if happened. By using two connections, you lost that ability by your self. I agree on this one. Could you expand a bit on this point? While the connection creation process (usually) asymmetric, once the connection is built it's essentially the same state machine on both sides of the connection, and socket semantics with respect to the state machine are effectively identical. Application on both sides should be able to detect disconnect, monitor connection state using TCP_INFO, etc. What I meant was that there were cases when a receiver could not tell weather no data was coming or communication was interrupted. Once connection is established, a route is available between a server and a client. Let's say this route is broken for some reasons, i.e. someone unplugged a cable or a firewall started dropping or rejecting between these server and client, a sender may not notice as soon as it happens but at least, a sender knows a massages was not delivered right. On the other hand, receiver side does not have any idea that a message delivery failure has happened at all or for a while unless using heartbeat messages in upper layer. KEEP_ALIVE option seems to be implementation dependent such that you cannot assure TCP connection availability for every minute. This is generally considered a robustness property rather than a fragility issue, but yes: if you need a liveliness property for idle connections with TCP, it's something you have to implement at the application layer, and many protocols indeed do this. I don't see that this is an argument for using two TCP connections as opposed to one, however. If you're interested in alternative protocols, however, SCTP allows a number of these protocol behaviors to be modified, and includes support for a heartbeat. Robert N M Watson Computer Laboratory University of Cambridge ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good?
On Fri, 20 Mar 2009 13:24:09 + (GMT) Robert Watson rwat...@freebsd.org wrote: On Fri, 20 Mar 2009, Yoshihiro Ota wrote: 1. With TCP connections, only sender side can detect some communication issues passively if happened. By using two connections, you lost that ability by your self. I agree on this one. Could you expand a bit on this point? While the connection creation process (usually) asymmetric, once the connection is built it's essentially the same state machine on both sides of the connection, and socket semantics with respect to the state machine are effectively identical. Application on both sides should be able to detect disconnect, monitor connection state using TCP_INFO, etc. What I meant was that there were cases when a receiver could not tell weather no data was coming or communication was interrupted. Once connection is established, a route is available between a server and a client. Let's say this route is broken for some reasons, i.e. someone unplugged a cable or a firewall started dropping or rejecting between these server and client, a sender may not notice as soon as it happens but at least, a sender knows a massages was not delivered right. On the other hand, receiver side does not have any idea that a message delivery failure has happened at all or for a while unless using heartbeat messages in upper layer. KEEP_ALIVE option seems to be implementation dependent such that you cannot assure TCP connection availability for every minute. Thanks, Hiro ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
2 uni-directional TCP connection good?
Hi forks. I have question on network programming. It will be nice if some could answer. I saw a program that opens 2 TCP connections. One connection is only used for server to client messaging only and the other connection is used only for client to server messaging. First of all, because TCP is already bi-directional communication, I don't think it is unnecessary to make 2 connection in the first place. After talking to my friend, he said it was very bad to do such things for three reasons. 1. With TCP connections, only sender side can detect some communication issues passively if happened. By using two connections, you lost that ability by your self. I agree on this one. 2. He also said that it would also waste network bandwidth. 3. He also said that it would causes some data flushing/synchronization issues. Indeed, this was what I saw with the program. However, I couldn't understand why it could happen. What I saw was from time to time, the sender side reported it send messages with some sequence numbers but the receiver didn't actually receive these messages for a long time, I think it was about a couple of seconds to several seconds between two hosts on the same switch. Could anyone explain if #2 is true and why #3 happens? Regards, Hiro ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good?
Yoshihiro Ota wrote: I saw a program that opens 2 TCP connections. One connection is only used for server to client messaging only and the other connection is used only for client to server messaging. 2. He also said that it would also waste network bandwidth. You have a two-way communication no matter what you do. But if you don't actually use inbound direction, all it gets used for is the receipt of ACK packets. That is, the inbound connection is used to make the data transfer reliable. If you don't have any payload data on the inbound connection, then the outbound connection won't have any ACK packets. If you're sending payload data, the ACK info can hitchhike along with the payload packets, thus saving bandwidth. But if you're not sending any payload data at all, there will be packets transmitted which contain the ACKs and nothing else. The extra network overhead will be modest if you're sending a lot of data all at once, say transferring a large file. But if very little data is sent per packet, say individual characters in a telnet connection, the overhead would be very high. If you have a single connection with payload data in both directions, then the ACKs will almost always ride along with some payload data. The only time a packet will contain nothing but an ACK will be when some data was transmitted, but none is to be received at the time. Mike -- Michael David Crawford m...@prgmr.com prgmr.com - We Don't Assume You Are Stupid. Xen-Powered Virtual Private Servers: http://prgmr.com/xen ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: 2 uni-directional TCP connection good?
On Fri, 20 Mar 2009, Yoshihiro Ota wrote: 1. With TCP connections, only sender side can detect some communication issues passively if happened. By using two connections, you lost that ability by your self. I agree on this one. Could you expand a bit on this point? While the connection creation process (usualy) asymetric, once the connection is built it's essentially the same state machine on both sides of the connection, and socket semantics with respect to the state machine are effectively identical. Application on both sides should be able to detect disconnect, monitor connection state using TCP_INFO, etc. Robert N M Watson Computer Laboratory University of Cambridge ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
