Hello Willy et al, I read the following blog post, published one year ago, about haproxy dropping connections waiting in the kernel listen queue during a restart:
http://www.ninjasys.co.uk/linux/haproxy-load-balancer-testing/ I was especially interested by Willy comment: *"we're all well aware of the issue you're mentionning. Its cause is deep in the kernel, and I tried to address it but it has too many dependencies and does not seem to interest enough people to get a proper fix. The issue is that connections that were sent to the old process' listen queue and not yet delivered to the process will be killed as soon as the process stops listening or goes away. I have tried to change this in several ways, for instance by transfering the few pending connections from one socket's queue to another one, but this touches too many areas which go as far as the file system."* With the current restart mechanism, where the old and the new process are independent and do not share socket file descriptors, the connections waiting in the listen queue of the old process are dropped when the process exits. This is the expected behavior. But I don't understand the following paragraph: *"At one point I thought that the principle of passing the listening FD between one process and another one would solve the issue, but unfortunately it does not for the same reason as above : at one point the old process stops listening so the requests pending in its queue get dropped."* Willy suggests that even when passing the FD from the old process to the new one, connections waiting in the listen queue are still dropped. I don't understand why. Under Linux, I can see two ways to pass and share the listening socket file descriptors: - The usual one is fork/exec. For example, the haproxy process could use execve to upgrade to a new binary and inherit listening socket file descriptors. uWSGI does this: http://uwsgi-docs.readthedocs.org/en/latest/Management.html#reloading-the-server - A less usual way is to launch the new process and pass the socket file descriptors through a Unix domain socket using SCM_RIGHTS. More clumsy in my opinion :) Example here: http://www.mail-archive.com/[email protected]/msg00002.html In both cases, the file descriptor (which is an integer) is duplicated in the old and new processes (in the process file descriptor table), but the file description is shared (in the kernel system-wide data structures). Because of this, I expect to have just one listen backlog, shared by both processes. If the file description and the listen backlog is shared, I don't understand why terminating the old process would drop connections waiting in the backlog. The new process is still supposed to accept them later? Am I missing something? Anyway, thanks for giving us haproxy! -- Nicolas Grilly Vocation City | Web Recruitment Platform Garden | Web Software Architects +33 1 45 72 48 78 - office +33 6 03 00 25 34 - mobile www.vocationcity.com
