I was also puzzled that binding succeeded. Looking into the code paths involved, in inet_csk_get_port, we quickly goto have_snum. From there, we end up dropping into tb_found. Since !hlist_empty(&tb->owners), we end up checking that (tb->fastreuseport > 0 && sk->sk_reuseport && uid_eq(tb->fastuid, uid)). This test passes, so we goto success and bind.
Crucially, we are checking the fastreuseport field on the inet_bind_bucket, and not the sk_reuseport variable on the other sockets in the bucket. Since this bit is set based on sk_reuseport at the time the first socket binds (see tb_not_found), I can see no reason why sockets need to keep SO_REUSEPORT set beyond initial binding. Given this, I believe Willy's patch elegantly solves the problem at hand. Josh On Wed, Dec 16, 2015 at 8:15 AM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Eric, > > On Wed, Dec 16, 2015 at 08:38:14AM +0100, Willy Tarreau wrote: >> On Tue, Dec 15, 2015 at 01:21:15PM -0800, Eric Dumazet wrote: >> > On Tue, 2015-12-15 at 20:44 +0100, Willy Tarreau wrote: >> > >> > > Thus do you think it's worth adding a new option as Tolga proposed ? >> > >> > >> > I thought we tried hard to avoid adding the option but determined >> > we could not avoid it ;) >> >> Not yet, your other proposal of disabling SO_REUSEPORT makes sense if >> we combine it with the proposal to change the score in my patch. If >> we say that a socket which has SO_REUSEPORT scores higher, then the >> connections which don't want to accept new connections anymore will >> simply have to drop it an not be elected. I find this even cleaner >> since the sole purpose of the loop is to find the best socket in case >> of SO_REUSEPORT. > > So I tried this and am pretty satisfied with the results, as I couldn't > see any single reset on 4.4-rc5 with it. On 4.1 I got a few very rare > resets at the exact moment the new process binds to the socket, because > I suspect some ACKs end up in the wrong queue exactly there. But > apparently the changes you did in 4.4 totally got rid of this, which is > great! > > I suspected that I could enter a situation where a new process could > fail to bind if generations n-1 and n-2 were still present, because > n-2 would be running without SO_REUSEPORT and that should make this > test fail in inet_csk_bind_conflict(), but it never failed for me : > > if ((!reuse || !sk2->sk_reuse || > sk2->sk_state == TCP_LISTEN) && > (!reuseport || !sk2->sk_reuseport || > (sk2->sk_state != TCP_TIME_WAIT && > !uid_eq(uid, sock_i_uid(sk2))))) { > ... > > So I'm clearly missing something and can't spot what. I mean, I'd > prefer to see my patch occasionally fail than not understanding why > it always works! If anyone has an suggestion I'm interested. > > Here's the updated patch. > > Best regards, > Willy > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html