Hi Willy and list, On Mon, Nov 18, 2013 at 11:32 AM, Willy Tarreau <[email protected]> wrote:
> > In both cases, the file descriptor (which is an integer) is duplicated in > > the old and new processes (in the process file descriptor table), but the > > file description is shared (in the kernel system-wide data structures). > > Because of this, I expect to have just one listen backlog, shared by both > > processes. If the file description and the listen backlog is shared, I > > don't understand why terminating the old process would drop connections > > waiting in the backlog. The new process is still supposed to accept them > > later? > > A totally agree with your analysis however I clearly have memories of > seeing something like a per-process queue in the kernel code when I > tracked this down. BTW, I also remember observing the same behaviour > when killing one process when running with nbproc > 1. > I tried to reproduce the issue with a small Python scripts that does the following: 1/ It starts a server process that listens on a TCP socket with a backlog of 128 (but do not accept connections). 2/ It starts a client process that opens 20 connections (using one thread per connection). 3/ Then the server process forks a new server process that inherits the listening socket, and exits. 4/ Then the new sever process starts to accept connections waiting in the accept queue. Everything works perfectly and no connection is dropped. I spend some time reading the TCP implementation of the kernel mainline (3.12) and found no trace of a per process accept queue. But I am an absolute newbie with the kernel networking code and it's very likely that I missed something. Regarding machines with a number of CPU/cores greater than 1, I have read many papers that propose to optimize the Linux kernel TCP code by partitioning the accept queue, with one queue per core, in order to avoid lock contention. But I don't know if it has been implemented in the kernel mainline. I'm really curious to know how you got those dropped connections with a shared listening socket. Because if it is true, it means the graceful reload advertised by a lot of open source projets (nginx being one of them [1]) is not really graceful and can still drop some established connections waiting in the accept queue! It would be really interesting to know :) [1] http://nginx.org/en/docs/control.html > Am I missing something? > > Maybe or maybe not. What kernel version did you check ? I think my tests > were somewhere between 2.6.32 and 3.5. > I tested using kernel version 3.2.0 and Ubuntu 12.04. Regards, Nicolas
