I've recently noticed that when using the FreeBSD TCP stack, the connect() system call ignores the SO_REUSEADDRESS setting and returns EADDRINUSE when attempting to connect a TCP socket if a connection with the same (SrcIP,SrcPort,DstIP,DstPort) tuple is still hanging around in the TIME_WAIT state.
Here's an example sequence of events: * create a TCP socket * set SO_REUSEADDRESS and SO_REUSEPORT options * bind to port 12346 * connect to 1.2.3.4:54321 * send a little data * close connection * wait a few seconds * create a TCP socket * set SO_REUSEADDRESS and SO_REUSEPORT options * bind() to port 12346 * connect() to 1.2.3.4:54321 The bind() will succeed due to the SO_REUSEADDR, SO_REUSEPORT option. The connect() will fail with EADDRINUSE. If you wait a minute or two between connections it works fine. There appear to be a certain set of conditions when an existing TIME_WAIT connection will be discarded and reused -- see line 747 below: >From tcp_usrreq.c: 733 /* 734 * Cannot simply call in_pcbconnect, because there might be an 735 * earlier incarnation of this same connection still in 736 * TIME_WAIT state, creating an ADDRINUSE error. 737 */ 738 error = in_pcbladdr(inp, nam, &ifaddr); 739 if (error) 740 return error; 741 oinp = in_pcblookup_hash(inp->inp_pcbinfo, 742 sin->sin_addr, sin->sin_port, 743 inp->inp_laddr.s_addr != INADDR_ANY ? inp->inp_laddr 744 : ifaddr->sin_addr, 745 inp->inp_lport, 0, NULL); 746 if (oinp) { 747 if (oinp != inp && (otp = intotcpcb(oinp)) != NULL && 748 otp->t_state == TCPS_TIME_WAIT && 749 (ticks - otp->t_starttime) < tcp_msl && 750 (otp->t_flags & TF_RCVD_CC)) 751 otp = tcp_close(otp); 752 else 753 return EADDRINUSE; 754 } But setting SO_REUSEADDR doesn't affect the behavior of connect(). Should it? -- Grant Edwards grant.b.edwards Yow! VICARIOUSLY experience at some reason to LIVE!! gmail.com -- Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss