Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Fri, Mar 09, 2001 at 02:56:21PM +0900, [EMAIL PROTECTED] wrote: > > >From the server's point of view: > > > >+ TCP/IP handshake from client, allocate protocol control blocks > >+ receive data from client > >+ client resets connection, pcb is destroyed > > > >Exactly why the client resets the connection isn't my concern at > >the moment. Some stacks may place a timeout on the FIN_WAIT state, > >and forcibly reset the reset the connection when the timer expires. > >Alternatively, the client may crash, and then RST in response to > >an ACK transmitted by the server. Or the other end may have set > >SO_LINGER, which will cause close() to send a RST. > > > >The unix-domain bug is because we were treating sockets in the > >DISCONNECTED state identically across all protocols, which turns > >out not to be the case. > > > >As for any data that already exists in the socket buffer on the > >server when the connection is aborted, I believe that the correct > >thing to do is discard it. This is the historical precedent, and > >is supported by the current standards. > > > >Below is a patch that will fix the behavior for unix-domain sockets. > > from code inspection on netbsd, I guess you'd need to do something for > other sys/net* families. maybe we need another per-domain/per-socket > flag for this? That might be cleaner in the long run rather than putting the test into each protocol layer. However, since FreeBSD is coming up on a release, I don't want to make an immediate change. I've only changed tcp/tcp6 behavior. The only other protocols that I know of that might need changing are ipx and aal; these will probably return EINVAL instead of ECONNABORTED at the moment. I think that the correct long term behavior would be to make the peer name available to the socket regardless of the state of the protocol connection (either hold the protocol block, or copy the name up to the socket layer). Then, restore the previous behavior of always returning a valid fd for the socket, even if the peer is closed. This provides two things: 1. The application is able to get the address of the peer who connected (whowever briefly). I've seen some requests for this particular feature. Doing this fixes the bug with DNS and Apache, which assume the returned address is valid. 2. The app will still be able to retrieve any data that is sitting in the TCP socket buffers, which might be desired in the TCP case (just as it is in the unix-domain case) -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>From the server's point of view: > >+ TCP/IP handshake from client, allocate protocol control blocks >+ receive data from client >+ client resets connection, pcb is destroyed > >Exactly why the client resets the connection isn't my concern at >the moment. Some stacks may place a timeout on the FIN_WAIT state, >and forcibly reset the reset the connection when the timer expires. >Alternatively, the client may crash, and then RST in response to >an ACK transmitted by the server. Or the other end may have set >SO_LINGER, which will cause close() to send a RST. > >The unix-domain bug is because we were treating sockets in the >DISCONNECTED state identically across all protocols, which turns >out not to be the case. > >As for any data that already exists in the socket buffer on the >server when the connection is aborted, I believe that the correct >thing to do is discard it. This is the historical precedent, and >is supported by the current standards. > >Below is a patch that will fix the behavior for unix-domain sockets. from code inspection on netbsd, I guess you'd need to do something for other sys/net* families. maybe we need another per-domain/per-socket flag for this? itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Thu, Mar 08, 2001 at 02:30:23PM -0800, Alfred Perlstein wrote: > * Jonathan Lemon <[EMAIL PROTECTED]> [010308 14:19] wrote: > > On Thu, Mar 08, 2001 at 01:00:48PM -0500, Wietse Venema wrote: > > > Jonathan Lemon: > > > > On Thu, Mar 08, 2001 at 10:38:17AM -0500, Wietse Venema wrote: > > > > > If the result of connect() write() close() depends on whether > > > > > accept() happens after or before close(), then the behavior is > > > > > broken. The client has received a successful return from write() > > > > > and close(). The system is not supposed to lose the data, period. > > > > > > > > What you seem to be missing here is that the behavior described > > > > above is ONLY specific to UNIX-DOMAIN sockets. The description > > > > above is generally (but not always) true for the TCP/IP protocol. > > > > > > The problem is observed with UNIX-domain sockets. > > > > > > > Data CAN be lost if the TCP connection is RST. It has nothing to > > > > do with the ordering of accept() with respect to close(). > > > > > > Please educate me: how would RST come into this discussion at all? > > > The client does connect() write() close(), there is no forced > > > connection termination involved at all. > > > > Under normal circumstances, a connect(), write(), close() call > > should work. However, the code that was added was to handle the > > abnormal cases from the server's point of view. > > Just make sure your patch is ok with the unix file descriptor > passing garbage collection code, it seems to rely on socket state > to get things right. It doesn't change any of the socket state, nor does it touch the SS_NOFDREF flag, so the gc algorithm should be okay. Actually, all I'm doing here is pushing the decision down to each protocol, rather than short-circuiting the test at the socket layer. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
The graceful-close debate is a very old one, going back more than twenty years. X.25 and ISO-TP have non-graceful close - the close can pass data in the network and cause it to be lost. TCP is defined as graceful-close. In SVR4 TLI there are two types of stream "sockets" with graceful or ugly close semantics. Having said all that, I certainly agree with Wietse that the POLA demands that a PF-LOCAL stream socket behave like TCP. Barney Wolff To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
* Jonathan Lemon <[EMAIL PROTECTED]> [010308 14:19] wrote: > On Thu, Mar 08, 2001 at 01:00:48PM -0500, Wietse Venema wrote: > > Jonathan Lemon: > > > On Thu, Mar 08, 2001 at 10:38:17AM -0500, Wietse Venema wrote: > > > > If the result of connect() write() close() depends on whether > > > > accept() happens after or before close(), then the behavior is > > > > broken. The client has received a successful return from write() > > > > and close(). The system is not supposed to lose the data, period. > > > > > > What you seem to be missing here is that the behavior described > > > above is ONLY specific to UNIX-DOMAIN sockets. The description > > > above is generally (but not always) true for the TCP/IP protocol. > > > > The problem is observed with UNIX-domain sockets. > > > > > Data CAN be lost if the TCP connection is RST. It has nothing to > > > do with the ordering of accept() with respect to close(). > > > > Please educate me: how would RST come into this discussion at all? > > The client does connect() write() close(), there is no forced > > connection termination involved at all. > > Under normal circumstances, a connect(), write(), close() call > should work. However, the code that was added was to handle the > abnormal cases from the server's point of view. Just make sure your patch is ok with the unix file descriptor passing garbage collection code, it seems to rely on socket state to get things right. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Thu, Mar 08, 2001 at 01:00:48PM -0500, Wietse Venema wrote: > Jonathan Lemon: > > On Thu, Mar 08, 2001 at 10:38:17AM -0500, Wietse Venema wrote: > > > If the result of connect() write() close() depends on whether > > > accept() happens after or before close(), then the behavior is > > > broken. The client has received a successful return from write() > > > and close(). The system is not supposed to lose the data, period. > > > > What you seem to be missing here is that the behavior described > > above is ONLY specific to UNIX-DOMAIN sockets. The description > > above is generally (but not always) true for the TCP/IP protocol. > > The problem is observed with UNIX-domain sockets. > > > Data CAN be lost if the TCP connection is RST. It has nothing to > > do with the ordering of accept() with respect to close(). > > Please educate me: how would RST come into this discussion at all? > The client does connect() write() close(), there is no forced > connection termination involved at all. Under normal circumstances, a connect(), write(), close() call should work. However, the code that was added was to handle the abnormal cases from the server's point of view. As you noted, this happened to break for unix-domain sockets under 4.2-stable, because of the following kernel semantics bug: + with unix-domain sockets, the connection is marked as DISCONNECTED as soon as the final close() is performed. + with TCP/IP sockets, a connection is marked "DISCONNECTING" on the final client close, but is NOT actually closed (marked as DISCONNECTED) until the server is notified that client's TCP/IP endpoint is gone. What we are trying to fix here is when the server, for some reason, happens to see the client forcibly tear down the endpoint before it can get around to to accepting the connection. >From the server's point of view: + TCP/IP handshake from client, allocate protocol control blocks + receive data from client + client resets connection, pcb is destroyed Exactly why the client resets the connection isn't my concern at the moment. Some stacks may place a timeout on the FIN_WAIT state, and forcibly reset the reset the connection when the timer expires. Alternatively, the client may crash, and then RST in response to an ACK transmitted by the server. Or the other end may have set SO_LINGER, which will cause close() to send a RST. The unix-domain bug is because we were treating sockets in the DISCONNECTED state identically across all protocols, which turns out not to be the case. As for any data that already exists in the socket buffer on the server when the connection is aborted, I believe that the correct thing to do is discard it. This is the historical precedent, and is supported by the current standards. Below is a patch that will fix the behavior for unix-domain sockets. -- Jonathan Index: kern/uipc_socket.c === RCS file: /ncvs/src/sys/kern/uipc_socket.c,v retrieving revision 1.68.2.13 diff -u -r1.68.2.13 uipc_socket.c --- kern/uipc_socket.c 2001/02/26 04:23:16 1.68.2.13 +++ kern/uipc_socket.c 2001/03/08 02:34:00 @@ -360,10 +360,7 @@ if ((so->so_state & SS_NOFDREF) == 0) panic("soaccept: !NOFDREF"); so->so_state &= ~SS_NOFDREF; - if ((so->so_state & SS_ISDISCONNECTED) == 0) - error = (*so->so_proto->pr_usrreqs->pru_accept)(so, nam); - else - error = ECONNABORTED; + error = (*so->so_proto->pr_usrreqs->pru_accept)(so, nam); splx(s); return (error); } Index: netinet/tcp_usrreq.c === RCS file: /ncvs/src/sys/netinet/tcp_usrreq.c,v retrieving revision 1.51 diff -u -r1.51 tcp_usrreq.c --- netinet/tcp_usrreq.c2000/01/09 19:17:28 1.51 +++ netinet/tcp_usrreq.c2001/03/08 16:21:28 @@ -417,6 +417,10 @@ struct inpcb *inp = sotoinpcb(so); struct tcpcb *tp; + if (so->so_state & SS_ISDISCONNECTED) { + error = ECONNABORTED; + goto out; + } COMMON_START(); in_setpeeraddr(so, nam); COMMON_END(PRU_ACCEPT); @@ -431,6 +435,10 @@ struct inpcb *inp = sotoinpcb(so); struct tcpcb *tp; + if (so->so_state & SS_ISDISCONNECTED) { + error = ECONNABORTED; + goto out; + } COMMON_START(); in6_mapped_peeraddr(so, nam); COMMON_END(PRU_ACCEPT); To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
RE: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
I would like to preempt corrections to the effect that it is currently impossible for accept to return both an error code and a socket to read the data from. It sounds like there may be a bug in the behavior of accept w.r.t Unix Domain sockets. For TCP, if the client sends data, then closes with SO_LINGER sending a RST, and then the server accepts, it is appropriate to return an error code and even lose the sent data. However, if the client simply sends a FIN (indicating that it will send no more data), then the connection should definitely accept without error, since the connection is only half-closed, and the server can send data to the client (in this case of accepting a half-closed connection, for kevent, a read filter should mark EV_EOF, and a write filter should not) > Let's agree: it is okay for accept to return an error code indicating the > connection has already been terminated, so long as any data sent by the client > (such that the client had every indication that it was received) is still > available for the acceptor to read. > > -Jon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
RE: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
> > Data CAN be lost if the TCP connection is RST. It has nothing to > > do with the ordering of accept() with respect to close(). > > Please educate me: how would RST come into this discussion at all? > The client does connect() write() close(), there is no forced > connection termination involved at all. > > Wietse If you set the SO_LINGER socket option, a close() may generate RST and discard socket buffers/TCP state, as opposed to the standard behavior of keeping the data around and resending until the data and the FIN are acknowleged by the other end. I am not sure why this is supposed to be a good idea, though, but obviously, if you set that option, you are unconcerned about data loss, or you have already guarded against it with application-level acknowledgment. Let's agree: it is okay for accept to return an error code indicating the connection has already been terminated, so long as any data sent by the client (such that the client had every indication that it was received) is still available for the acceptor to read. -Jon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Thu, Mar 08, 2001 at 10:38:17AM -0500, Wietse Venema wrote: > If the result of connect() write() close() depends on whether > accept() happens after or before close(), then the behavior is > broken. The client has received a successful return from write() > and close(). The system is not supposed to lose the data, period. What you seem to be missing here is that the behavior described above is ONLY specific to UNIX-DOMAIN sockets. The description above is generally (but not always) true for the TCP/IP protocol. Data CAN be lost if the TCP connection is RST. It has nothing to do with the ordering of accept() with respect to close(). -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Thu, Mar 08, 2001 at 12:52:31PM +0900, [EMAIL PROTECTED] wrote: > > >Several parts of Postfix do: connect() write() close(), where the > >close() may happen before the server has accept()ed the connection. > >Due to an incompatible change in FreeBSD 4.2-STABLE, this causes > >accept() after close() to fail. The already written data is lost. > >This is a bad incompatible change. > > see SUSv2 and XNET 5.2 spec for accept(), and Stevens (unix network > programming vol 1 2nd ed) section 5.11. > http://www.opengroup.org/onlinepubs/007908799/xns/accept.html > > 4.3BSD behavior was very wrong (described in Stevens). > > old freebsd behavior (return 0-length sockaddr) was also wrong, > it kills a set of applications including sendmail and BIND910. > > new freebsd behavior (ECONNABORTED) is at least SUSv2 conformant, > and I expect it to be the behavior of other stacks. The crucial part of Wietse's message (which he didn't mention in the first email) is that he's using this setup on unix-domain sockets. These probably have semantics which are slightly different. I believe the correct thing to do at this point is push the IS_DISCONNECTED test down to each protocol's accept routine, instead of short-circuiting it in soaccept. It looks like the short-circuit test was added in uipc_socket.c:1.41 in the NetBSD sources, which we brought over. I'm not sure why this test is actually needed, though. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>Several parts of Postfix do: connect() write() close(), where the >close() may happen before the server has accept()ed the connection. >Due to an incompatible change in FreeBSD 4.2-STABLE, this causes >accept() after close() to fail. The already written data is lost. >This is a bad incompatible change. see SUSv2 and XNET 5.2 spec for accept(), and Stevens (unix network programming vol 1 2nd ed) section 5.11. http://www.opengroup.org/onlinepubs/007908799/xns/accept.html 4.3BSD behavior was very wrong (described in Stevens). old freebsd behavior (return 0-length sockaddr) was also wrong, it kills a set of applications including sendmail and BIND910. new freebsd behavior (ECONNABORTED) is at least SUSv2 conformant, and I expect it to be the behavior of other stacks. itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
In article [EMAIL PROTECTED]> you write: >In article ><[EMAIL PROTECTED]> Robert >Watson wrote: > >>It seems that the ECONNABORTED is the "standard" way to address this >>scenario; it might be an interesting exercise for someone to look at the >>common application suites and see how they respond to various failure >>modes in accept(). It certainly appears that BIND9 expects that. > >FYI, Postfix seems to have problems with the ECONNABORTED part of the >change, see the mail below from postfix-users. > >Arjan > >-- >Arjan de Vet, Eindhoven, The Netherlands <[EMAIL PROTECTED]> >URL: http://www.iae.nl/users/devet/ for PGP key: finger [EMAIL PROTECTED] > >Subject: Re: errors from flush [again] >In-Reply-To: <[EMAIL PROTECTED]> "from >Wietse Venema at Mar 6, 2001 06:49:57 pm" >To: Wietse Venema <[EMAIL PROTECTED]> >Date: Wed, 7 Mar 2001 10:26:22 -0500 (EST) >Cc: Arjan de Vet <[EMAIL PROTECTED]>, [EMAIL PROTECTED] >Message-ID: <[EMAIL PROTECTED]> >From: [EMAIL PROTECTED] (Wietse Venema) > >Postfix 20010228 is OK on FreeBSD 4.2-RELEASE. > >I just checked that the system behaves sanely when Postfix does > > connect() > write() > close() > >before the server has accept()ed the connection. > >Reportedly, some later FreeBSD versions introduce a race condition >here. If the server accept()s the connection before the client >close()s it, then the server may receive the data. If the server >accept()s the connection after the client close()s it, the data is >lost. In both cases, the client is told that the data was transmitted >without error. Actually, this is incorrect. The server in both cases will correctly receive the data from the client; it does not matter if the client has (correctly) closed the socket before the server gets around to accepting it. This case only applies to when connection is reset after the initial handshake, but before the connection is accepted. All we are doing is returning an error immediately via accept(), instead of waiting for a subsequent operation on the socket to fail. There is no change to the read/write semantics here; the local TCP endpoint (and any buffers) have already been destroyed. I probably poorly worded the commit message; it should read "return ECONNABORTED if connection receives a RST while on the listen queue". Under normal operation, the client's connection isn't really closed until it receives a FIN/ACK from the server. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
In article <[EMAIL PROTECTED]> Robert Watson wrote: >It seems that the ECONNABORTED is the "standard" way to address this >scenario; it might be an interesting exercise for someone to look at the >common application suites and see how they respond to various failure >modes in accept(). It certainly appears that BIND9 expects that. FYI, Postfix seems to have problems with the ECONNABORTED part of the change, see the mail below from postfix-users. Arjan -- Arjan de Vet, Eindhoven, The Netherlands <[EMAIL PROTECTED]> URL: http://www.iae.nl/users/devet/ for PGP key: finger [EMAIL PROTECTED] Subject: Re: errors from flush [again] In-Reply-To: <[EMAIL PROTECTED]> "from Wietse Venema at Mar 6, 2001 06:49:57 pm" To: Wietse Venema <[EMAIL PROTECTED]> Date: Wed, 7 Mar 2001 10:26:22 -0500 (EST) Cc: Arjan de Vet <[EMAIL PROTECTED]>, [EMAIL PROTECTED] Message-ID: <[EMAIL PROTECTED]> From: [EMAIL PROTECTED] (Wietse Venema) Postfix 20010228 is OK on FreeBSD 4.2-RELEASE. I just checked that the system behaves sanely when Postfix does connect() write() close() before the server has accept()ed the connection. Reportedly, some later FreeBSD versions introduce a race condition here. If the server accept()s the connection before the client close()s it, then the server may receive the data. If the server accept()s the connection after the client close()s it, the data is lost. In both cases, the client is told that the data was transmitted without error. This change in socket behavior breaks the Postfix flush service, and probably also breaks other parts of Postfix as well. It all depends on how quickly a server can call accept(), and thus, it all depends on how loaded a server is. This is a bad change. uname -a output: FreeBSD hades.porcupine.org 4.2-RELEASE FreeBSD 4.2-RELEASE #5: Sun Mar 4 18:58:52 EST 2001 [EMAIL PROTECTED]:/usr/src/sys/compile/GENERIC i386 Wietse Wietse Venema: > That's it. Postfix connects/writes/disconnects. The changed behavior > causes loss of data that was already written and acknowledged. > > The bad part is that there is no way for the sender to find out. > > Wietse > > Arjan de Vet: > > In article <[EMAIL PROTECTED]> you write: > > > > >Every 100 seconds means whenever the master 'wakeup' timer goes > > >off. This means that UNIX-domain socket behavior has changed. > > > > I've looked through recent commits to socket code and this one may be > > the problem (not verified yet, it's time to go to sleep ;-): > > > > jlemon 2001/02/13 18:09:12 PST > > > > Modified files: > > sys/kern uipc_socket.c uipc_syscalls.c > > Log: > > Return ECONNABORTED from accept if connection is closed while on the > > listen queue, as well as the current behavior of a zero-length sockaddr. > > > > Obtained from: KAME > > Reviewed by: -net > > > > Revision ChangesPath > > 1.88 +3 -6 src/sys/kern/uipc_socket.c > > 1.85 +15 -2 src/sys/kern/uipc_syscalls.c > > > > This change was applied on Feb 24 to FreeBSD 4.2-stable. > > > > The change in uipc_socket.c is: > > > > @@ -365,11 +365,8 @@ > > so->so_state &= ~SS_NOFDREF; > > if ((so->so_state & SS_ISDISCONNECTED) == 0) > > error = (*so->so_proto->pr_usrreqs->pru_accept)(so, > > nam); > > - else { > > - if (nam) > > - *nam = 0; > > - error = 0; > > - } > > + else > > + error = ECONNABORTED; > > splx(s); > > return (error); > > } > > > > ECONNABORTED is exactly the error I'm seeing now. > > > > >At the very least, the kernel could deliver the data that > > >is send with connect/write/disconnect. > > > > Arjan > > > > -- > > Arjan de Vet, Eindhoven, The Netherlands <[EMAIL PROTECTED]> > > URL: http://www.iae.nl/users/devet/ for PGP key: finger [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>> for background (like when this happens) see previous articles >> on this thread. >> current behavior: return 0-length sockaddr. >Yeah, that is totally broken. >Hmm.. how long has this been the "current behavior" ? >ISTR at one time you would instead get the actual sockaddr of the >just-closed socket, rather than a bogus sockaddr... and that is the >behavior one would expect. No, i guess your memory is wrong (or remember something different). Before sys/kern/uipc_socket.c 1.52, 4.4BSD-based systems hanged up when: - TCP handshake is done - RST is issued before accept(2) after 1.52 to current, zero-length sockaddr is returned. none of these are correct. >> new behavior: return ECONNABORTED. >> SUSv2 suggests this behavior. it is much safer as accept(2) will fail >> so almost every application will go to error case (if you don't have >> error check in userland appication, that's problem in application). >Why does SUSv2 suggest this when so many applications would break? >And they work fine doing the old behavior (returning a real sockaddr)? again, *BSD never worked right (in terms of SUSv2 definition of "right"). see Stevens "unix network programming" section I mentioned earlier in this thread. itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Mon, Feb 12, 2001 at 06:34:29PM -0800, Archie Cobbs wrote: > Jonathan Lemon writes: > > It seems to me that the odds of an application being able to correctly > > handle an error return from accept() are far greater than the odds that > > the code correctly checks 'len' upon return from accept. This, combined > > with the standard, seems to be rationale enough to make the change. > > True.. but even greater still are the chances that an application will > correctly handle accept() returning a *real* sockaddr that corresponds > to an already closed connection. Please review the earlier postings in this thread. While this would probably be nice, it isn't possible with our current internals; the sockaddr is stored in the inpcb, which is destroyed when the connection is torn down. So by the time the user gets around to calling accept, the sockaddr is long gone. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
> struct sockaddr_in sin; > > len = sizeof(sin); > fd = accept(s, (struct sockaddr *)&sin, &len); > if (fd == -1) > err(1, "accept"); > printf("peer address: %s\n", inet_ntoa(sin.sin_addr)); > >The bug with this code is that it blindly uses ``sin'' after accept >returns, since the usage of the returned sockaddr must first be qualified >by checking the returned value of len. Current behavior is for accept >to return a valid fd, and set len to 0. > >To my knowledge, most applications (certainly those I inspected) do not >bother to check 'len' before using 'sin', and thus do the wrong thing; >this is the bug that BIND ran into. (just to be clear, this is a bug >in the application code, not the kernel; although you could make the >case that it is a documentation bug as well). I wonder if we can call it an "application bug". yes, it was application's fault that it did not check sa_len. from the text below: http://www.opengroup.org/onlinepubs/007908799/xns/accept.html i'm not sure if it is legal to return zero-length sockaddr when the kernel is given enough space to return the sockaddr. (the last paragraph do not fit to TCP) itojun If address is not a null pointer, the address of the peer for the accepted connection is stored in the sockaddr structure pointed to by address, and the length of this address is stored in the object pointed to by address_len. If the actual length of the address is greater than the length of the supplied sockaddr structure, the stored address will be truncated. If the protocol permits connections by unbound clients, and the peer is not bound, then the value stored in the object pointed to by address is unspecified. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Mon, Feb 12, 2001 at 04:51:48PM -0800, Archie Cobbs wrote: > Jonathan Lemon writes: > > > > Did you guys agree on a commit-worthy fix yet? > > > > > > I wasn't party to the issue that generated this thread in the first > > > place, but.. I think the concensus is that if accept(2) returns > > > an error then this will break some applications, so instead it > > > should return a socket which will itself return an error on the > > > first operation. Somebody correct me if I'm wrong. > > > > No, as this is the current behavior. The change will be for accept > > to return an error, on the basis that 1) most apps already do the > > wrong thing now anyway, and 2) it brings us closer to a 'standard', > > e.g.: what other systems are doing as well. > > I don't understand then. > > What is the problem with the current behavior? Is this just an > optimization or a real bug fix? I'd say it's not worth changing > if it's just an optimization, because too many things will break. > Several apps have already been pointed out. > > And what do you mean by ``most apps already do the wrong thing now''? Consider the following bit of "canonical" code, of which some variant is probably found in almost all network programs: struct sockaddr_in sin; len = sizeof(sin); fd = accept(s, (struct sockaddr *)&sin, &len); if (fd == -1) err(1, "accept"); printf("peer address: %s\n", inet_ntoa(sin.sin_addr)); The bug with this code is that it blindly uses ``sin'' after accept returns, since the usage of the returned sockaddr must first be qualified by checking the returned value of len. Current behavior is for accept to return a valid fd, and set len to 0. To my knowledge, most applications (certainly those I inspected) do not bother to check 'len' before using 'sin', and thus do the wrong thing; this is the bug that BIND ran into. (just to be clear, this is a bug in the application code, not the kernel; although you could make the case that it is a documentation bug as well). It seems that other systems (and maybe the SuS?) expect accept to return ECONNABORTED in this case. Both BIND and Apache expect this, at least. It seems to me that the odds of an application being able to correctly handle an error return from accept() are far greater than the odds that the code correctly checks 'len' upon return from accept. This, combined with the standard, seems to be rationale enough to make the change. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>> No, as this is the current behavior. The change will be for accept >> to return an error, on the basis that 1) most apps already do the >> wrong thing now anyway, and 2) it brings us closer to a 'standard', >> e.g.: what other systems are doing as well. > >I don't understand then. > >What is the problem with the current behavior? Is this just an >optimization or a real bug fix? I'd say it's not worth changing >if it's just an optimization, because too many things will break. >Several apps have already been pointed out. > >And what do you mean by ``most apps already do the wrong thing now''? for background (like when this happens) see previous articles on this thread. current behavior: return 0-length sockaddr. many of the existing applications assumes to get valid sockaddr to the address family of the socket (like sockaddr_in for AF_INET socket). so if they do that, they will access uninitialized memory region, die with assert(), whatever (BIND 9.1.0 dies with assert()). it is rather questionable if the kernel can return 0-length sockaddr, standard-wise, at least for connection-oriented socket. new behavior: return ECONNABORTED. SUSv2 suggests this behavior. it is much safer as accept(2) will fail so almost every application will go to error case (if you don't have error check in userland appication, that's problem in application). itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Mon, Feb 12, 2001 at 09:56:26AM -0800, Archie Cobbs wrote: > Kris Kennaway writes: > > Did you guys agree on a commit-worthy fix yet? > > I wasn't party to the issue that generated this thread in the first > place, but.. I think the concensus is that if accept(2) returns > an error then this will break some applications, so instead it > should return a socket which will itself return an error on the > first operation. Somebody correct me if I'm wrong. No, as this is the current behavior. The change will be for accept to return an error, on the basis that 1) most apps already do the wrong thing now anyway, and 2) it brings us closer to a 'standard', e.g.: what other systems are doing as well. I was planning on committing the change soon, but there isn't really any hurry; effectively all it does is change the mechanism of error reporting from one form to another. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Thu, Feb 08, 2001 at 10:12:43AM -0800, Archie Cobbs wrote: > Jonathan Lemon writes: > > >> Jayanth did make one point that an application could assume that > > >> the error return from accept was in regards to the listening socket > > >> instead of the new socket, so that may be a concern. > > > > > >Yes I have always assumed this to be true. If the connection is > > >already broken before I get it, why bother giving it to me?? > > > > Because the connection may be broken between the point where we've > > notified the user that there is a valid connection, and when accept > > returns. E.g.: > > It was a rhetorical question. I like Robert's suggestion to return > the socket but have the first operation on the socket fail. > > If you want to positively verify the new socket you can do a > getpeername(), etc. > > > I would guess that code is more likely to check for an error > > return from accept() than it is to check the return size from > > the sockaddr, so this change will proabably not result in breaking > > a large amount of code. > > IMHO the app is more likely to close the listen socket and stop > accepting new connections if it gets an error from accept(). > > For example, from an initial scan of sendmail (daemon.c:403) if > it gets an error from accept(2) it will close the (listen) socket > and reopen it. Sendmail is robust enough not to "break" but this > shows that if it gets an accept(2) error it assumes the problem is > with the *listening* socket, not the new socket. Yes, I looked at sendmail, ftpd, sshd, telnetd, inetd, and apache. Only apache does the right thing in this case. All the other applications either: 1. handle the error from accept() in some form (which may include terminating the application), or 2. blindly use the returned sockname. So in sendmail's case, it currently does: t = accept(.., (struct sockaddr *)&RealHostAddr, &lotherend); p = hostnamebyanyaddr(&RealHostAddr); which will proabably coredump since it dereferences a NULL pointer. I'd think that at least having sendmail close/reopen the listen socket will result in slightly more robust behavior. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
In article [EMAIL PROTECTED]> you write: > >On Wed, 7 Feb 2001, Archie Cobbs wrote: > >> Jonathan Lemon writes: >> > Jayanth did make one point that an application could assume that >> > the error return from accept was in regards to the listening socket >> > instead of the new socket, so that may be a concern. >> >> Yes I have always assumed this to be true. If the connection is >> already broken before I get it, why bother giving it to me?? > >Well, for the purposes of propagating information up to applications >consistently on both sides of a connection, the ideal behavior from my >perspective is: > > When a connection comes in and is reset/closed before accept() can > occur, accept() should return success with a properly filled out > sockaddr. When the first operation occurs on the socket, it should > return an appropriate error message based on the close of the socket > (ECONNRESET most likely). The difficulty with this is that the peer address is being held in the inpcb, which is released by tcp_close upon receipt of a RST. So at the moment, there isn't anywhere to retrieve the sockaddr from. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
On Wed, 7 Feb 2001, Archie Cobbs wrote: > Jonathan Lemon writes: > > Jayanth did make one point that an application could assume that > > the error return from accept was in regards to the listening socket > > instead of the new socket, so that may be a concern. > > Yes I have always assumed this to be true. If the connection is > already broken before I get it, why bother giving it to me?? Well, for the purposes of propagating information up to applications consistently on both sides of a connection, the ideal behavior from my perspective is: When a connection comes in and is reset/closed before accept() can occur, accept() should return success with a properly filled out sockaddr. When the first operation occurs on the socket, it should return an appropriate error message based on the close of the socket (ECONNRESET most likely). This allows the application to be notified of the event at the TCP level, log the address or the like, and continue. The only real issue with this behavior is that most applications assume that a successful accept() call returns a live socket, and may go through expensive overhead before their next socket operation -- for example, fork(). It seems that the ECONNABORTED is the "standard" way to address this scenario; it might be an interesting exercise for someone to look at the common application suites and see how they respond to various failure modes in accept(). It certainly appears that BIND9 expects that. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>> Jayanth did make one point that an application could assume that >> the error return from accept was in regards to the listening socket >> instead of the new socket, so that may be a concern. >Yes I have always assumed this to be true. If the connection is >already broken before I get it, why bother giving it to me?? can we cancel sorwakeup() (happens on handshake completion)? I believed not. itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
> i believe you will want to merge this. > scenario: > - you are listening to tcp port > - someone comes in, handshake (SYN, SYNACK, ACK) > - someone sends RST > - your server issues accept(2) > previous behavior: accept(2) returns successful result with zero- > length sockaddr. > new behavior: return ECONNABORTED. > > effect: > - if someone runs nmap against your machine, and you are unlucky, > your server listening to tcp port (like BIND9) can get > segv/abort due to unexpected zero-length sockaddr + successful > error return on accept(2). FYI: 9.1.0 had assert() against sockaddr returned by accept(2). therefore BIND 9.1.0 will get killed (or go suicide) by remote nmap with "previous (kernel) behavior" presented above. (it will only happen you are very unlucky - it is timing issue) BIND 9.1.1rc1 now includes workaround (no assert). itojun > 727. [port] Work around OS bug where accept() succeeds but >fails to fill in the peer address of the accepted >connection, by treating it as an error rather than >an assertion failure. [RT #809] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>Looks good to me (although the patch is mixed in with a bunch >of other crud). I've tested it out locally and will commit it >unless there are any objections. it is because of cvs issue. the important portion is below. itojun @@ -320,11 +359,8 @@ soaccept(so, nam) so->so_state &= ~SS_NOFDREF; if ((so->so_state & SS_ISDISCONNECTED) == 0) error = (*so->so_proto->pr_usrreqs->pru_accept)(so, nam); - else { - if (nam) - *nam = 0; - error = 0; - } + else + error = ECONNABORTED; splx(s); return (error); } To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>>>Looks good to me (although the patch is mixed in with a bunch >>>of other crud). I've tested it out locally and will commit it >>>unless there are any objections. >> it is because of cvs issue. the important portion is below. > oops, need some change in uipc_syscalls.c side... hold on. this diff is taken against 4.2-RELEASE (in kame tree), sorry. also i need to say that i ran no tests (i have no environment). uipc_syscalls.c change: make very sure to nuke file descriptor on errors uipc_socket.c change: return ECONNABORTED itojun Index: kern/uipc_socket.c === RCS file: /cvsroot/kame/kame/freebsd4/sys/kern/uipc_socket.c,v retrieving revision 1.1.1.3 retrieving revision 1.3 diff -u -r1.1.1.3 -r1.3 --- kern/uipc_socket.c 2001/01/19 02:25:59 1.1.1.3 +++ kern/uipc_socket.c 2001/02/08 01:37:42 1.3 @@ -362,7 +362,7 @@ else { if (nam) *nam = 0; - error = 0; + error = ECONNABORTED; } splx(s); return (error); Index: kern/uipc_syscalls.c === RCS file: /cvsroot/kame/kame/freebsd4/sys/kern/uipc_syscalls.c,v retrieving revision 1.1.1.3 retrieving revision 1.2 diff -u -r1.1.1.3 -r1.2 --- kern/uipc_syscalls.c2001/01/19 02:26:00 1.1.1.3 +++ kern/uipc_syscalls.c2001/02/08 01:37:42 1.2 @@ -270,28 +270,22 @@ fp->f_ops = &socketops; fp->f_type = DTYPE_SOCKET; sa = 0; - (void) soaccept(so, &sa); - if (sa == 0) { - namelen = 0; - if (uap->name) - goto gotnoname; - splx(s); - return 0; - } - if (uap->name) { + error = soaccept(so, &sa); + if (!error && uap->name) { /* check sa_len before it is destroyed */ - if (namelen > sa->sa_len) - namelen = sa->sa_len; + if (sa) { + if (namelen > sa->sa_len) + namelen = sa->sa_len; #ifdef COMPAT_OLDSOCK - if (compat) - ((struct osockaddr *)sa)->sa_family = - sa->sa_family; + if (compat) + ((struct osockaddr *)sa)->sa_family = + sa->sa_family; #endif - error = copyout(sa, (caddr_t)uap->name, (u_int)namelen); - if (!error) -gotnoname: - error = copyout((caddr_t)&namelen, - (caddr_t)uap->anamelen, sizeof (*uap->anamelen)); + error = copyout(sa, (caddr_t)uap->name, (u_int)namelen); + } else + namelen = 0; + error = copyout((caddr_t)&namelen, + (caddr_t)uap->anamelen, sizeof (*uap->anamelen)); } if (sa) FREE(sa, M_SONAME); To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
Current updated patch for comments is below. Jayanth did make one point that an application could assume that the error return from accept was in regards to the listening socket instead of the new socket, so that may be a concern. -- Jonathan Index: uipc_socket.c === RCS file: /ncvs/src/sys/kern/uipc_socket.c,v retrieving revision 1.68.2.11 diff -u -r1.68.2.11 uipc_socket.c --- uipc_socket.c 2000/12/22 10:25:21 1.68.2.11 +++ uipc_socket.c 2001/02/07 20:30:29 @@ -365,11 +365,8 @@ so->so_state &= ~SS_NOFDREF; if ((so->so_state & SS_ISDISCONNECTED) == 0) error = (*so->so_proto->pr_usrreqs->pru_accept)(so, nam); - else { - if (nam) - *nam = 0; - error = 0; - } + else + error = ECONNABORTED; splx(s); return (error); } Index: uipc_syscalls.c === RCS file: /ncvs/src/sys/kern/uipc_syscalls.c,v retrieving revision 1.65.2.6 diff -u -r1.65.2.6 uipc_syscalls.c --- uipc_syscalls.c 2000/12/02 00:47:50 1.65.2.6 +++ uipc_syscalls.c 2001/02/07 21:27:03 @@ -283,7 +283,19 @@ nfp->f_ops = &socketops; nfp->f_type = DTYPE_SOCKET; sa = 0; - (void) soaccept(so, &sa); + error = soaccept(so, &sa); + if (error) { + /* +* return a namelen of zero for older code which might +* ignore the return value from accept. +*/ + if (uap->name != NULL) { + namelen = 0; + (void) copyout((caddr_t)&namelen, + (caddr_t)uap->anamelen, sizeof(*uap->anamelen)); + } + goto noconnection; + } if (sa == NULL) { namelen = 0; if (uap->name) @@ -307,6 +319,7 @@ error = copyout((caddr_t)&namelen, (caddr_t)uap->anamelen, sizeof (*uap->anamelen)); } +noconnection: if (sa) FREE(sa, M_SONAME); To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>>Looks good to me (although the patch is mixed in with a bunch >>of other crud). I've tested it out locally and will commit it >>unless there are any objections. > it is because of cvs issue. the important portion is below. oops, need some change in uipc_syscalls.c side... hold on. itojun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
>>Looks good to me (although the patch is mixed in with a bunch >>of other crud). I've tested it out locally and will commit it >>unless there are any objections. > > it is because of cvs issue. the important portion is below. Looks good to me as well. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message
Re: [itojun@iijlab.net: accept(2) behavior with tcp RST right after handshake]
Won't comment on the implementation as I have't had a chance to review it yet, but the description sounds right, and compatible with http://www.opengroup.org/orc/DOCS/XNS/text/accept.htm http://www.fifi.org/cgi-bin/man2html/usr/share/man/man2/accept.2.gz There are some interesting comments with noting in a quote in http://www.humanfactor.com/cgi-bin/cgi-delegate/apache-ML/nh/1997/Jan/1176.html I hope to take a look at the implementation this evening. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services On Wed, 7 Feb 2001, Kris Kennaway wrote: > Can anyone comment on this patch? > > http://www.kame.net/dev/cvsweb.cgi/kame/freebsd4/sys/kern/uipc_socket.c > > Kris > > - Forwarded message from [EMAIL PROTECTED] - > > Delivered-To: [EMAIL PROTECTED] > Delivered-To: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > Subject: accept(2) behavior with tcp RST right after handshake > X-Template-Reply-To: [EMAIL PROTECTED] > X-Template-Return-Receipt-To: [EMAIL PROTECTED] > X-PGP-Fingerprint: F8 24 B4 2C 8C 98 57 FD 90 5F B4 60 79 54 16 E2 > From: [EMAIL PROTECTED] > Date: Wed, 07 Feb 2001 21:39:49 +0900 > X-UIDL: aff7d2fbee72775e2137abcde0bef0d0 > > i believe you will want to merge this. > scenario: > - you are listening to tcp port > - someone comes in, handshake (SYN, SYNACK, ACK) > - someone sends RST > - your server issues accept(2) > previous behavior: accept(2) returns successful result with zero- > length sockaddr. > new behavior: return ECONNABORTED. > > effect: > - if someone runs nmap against your machine, and you are unlucky, > your server listening to tcp port (like BIND9) can get > segv/abort due to unexpected zero-length sockaddr + successful > error return on accept(2). > > itojun > > --- Forwarded Messages > > Return-Path: [EMAIL PROTECTED] > Return-Path: <[EMAIL PROTECTED]> > Received: from orange.kame.net (orange.kame.net [203.178.141.194]) > by coconut.itojun.org (8.9.3+3.2W/3.7W) with ESMTP id VAA00242 > for <[EMAIL PROTECTED]>; Wed, 7 Feb 2001 21:35:16 +0900 (JST) > Received: (from daemon@localhost) > by orange.kame.net (8.9.3+3.2W/3.7W/smtpfeed 1.06) id VAA48429; > Wed, 7 Feb 2001 21:35:16 +0900 (JST) > Received: (from itojun@localhost) > by orange.kame.net (8.9.3+3.2W/3.7W) id VAA48423; > Wed, 7 Feb 2001 21:35:15 +0900 (JST) > Date: Wed, 7 Feb 2001 21:35:15 +0900 (JST) > From: Jun-ichiro itojun Hagino <[EMAIL PROTECTED]> > Message-Id: <[EMAIL PROTECTED]> > To: cvs-kame:; > Subject: kame cvs commit: kame/freebsd4/sys/kern uipc_socket.c kame/netbsd/sys/kern > uipc_socket.c kame/openbsd/sys/kern uipc_socket.c > Reply-to: [EMAIL PROTECTED] > X-Filter: mailagent [version 3.0 PL68] for [EMAIL PROTECTED] > > itojun 2001/02/07 21:35:15 JST > > Modified files: > freebsd4/sys/kernuipc_socket.c > netbsd/sys/kern uipc_socket.c > openbsd/sys/kern uipc_socket.c > Log: > return ECONNABORTED, if the socket (tcp connection for example) > is disconnected by RST right before accept(2). fixes PR 10698/12027. > checked with SUSv2, XNET 5.2, and Stevens (unix network programming > vol 1 2nd ed) section 5.11. > > Revision ChangesPath > 1.2 +243 -10 kame/freebsd4/sys/kern/uipc_socket.c > 1.3 +1 -1 kame/netbsd/sys/kern/uipc_socket.c > 1.3 +1 -1 kame/openbsd/sys/kern/uipc_socket.c > > --- Message 2 > > Return-Path: [EMAIL PROTECTED] > Return-Path: <[EMAIL PROTECTED]> > Received: from orange.kame.net (orange.kame.net [203.178.141.194]) > by coconut.itojun.org (8.9.3+3.2W/3.7W) with ESMTP id VAA00253 > for <[EMAIL PROTECTED]>; Wed, 7 Feb 2001 21:35:20 +0900 (JST) > Received: (from itojun@localhost) > by orange.kame.net (8.9.3+3.2W/3.7W/smtpfeed 1.06) id VAA48466; > Wed, 7 Feb 2001 21:35:19 +0900 (JST) > Date: Wed, 7 Feb 2001 21:35:19 +0900 (JST) > From: Jun-ichiro itojun Hagino <[EMAIL PROTECTED]> > Message-Id: <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Subject: kame-local cvs commit: kame/bsdi4/sys/kern uipc_socket.c > X-Filter: mailagent [version 3.0 PL68] for [EMAIL PROTECTED] > > itojun 2001/02/07 21:35:19 JST > > Modified files: > bsdi4/sys/kern uipc_socket.c > Log: > return ECONNABORTED, if the socket (tcp connection for example) > is disconnected by RST right before accept(2). fixes PR 10698/12027. > checked with SUSv2, XNET 5.2, and Stevens (unix network programming > vol 1 2nd ed) section 5.11. > > Revision ChangesPath > 1.4 +1 -1 kame/bsdi4/sys/kern/uipc_socket.c > > --- End of Forwarded Messages > > > > - End forwarded message - > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-net" in the body of the message