Attaching focal reproducer for SRU testing purposes. This was based on the autopkgtest suite available in impish
** Description changed: [Impact] Having the listen backlog (the TCP pending connections queue) set to 1, means that only one connection can be in a "waiting for an 'accept' call" state. If more connections are coming through in that window, i.e., before the connection in the queue is accepted, these additional connections will be ignored. While the short time a system usually takes to clear that queue combined with the possibility of a connection being reset and retried by a client will diminish the impact of the queue maximum size on the users, special cases where too many connections are being received in a short time span may result in many of those connection requests to either take a long time to be processed or to fail. One example of such cases is when the openvpn server is restarted. The proposed patch increases the listen backlog size to an arbitrary, larger size proposed and accepted upstream in https://community.openvpn.net/openvpn/changeset/ec0ca68f4ed1e6aa6f08f470b18e0198b7e5a4da [Test Plan] + * I am attaching scripts to reproduce/test the bug and the fix, since + the scripts may be a bit long. Below is a short summary on how to test + (describing the steps taken by the attached scripts). + - Configure and start an openvpn server - Perform several TCP connections to that server in a short time span, while measuring the time the whole connection batch takes to be accepted. You can also count the number of connections that get reset. - Apply the proposed patch - Verify how the time for the same operation described above is drastically reduced. You can also verify that the number of reset connections dropped (possibly to zero, depending on the amount of connections being performed). Here is a python script that could be used to perform the tests on a running server: ``` import socket import threading HOST = 'localhost' PORT = 1194 - def run(name): - for i in range(5): - s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - try: - s.connect((HOST, PORT)) - s.sendall(b'hello') - data = s.recv(1024) - except ConnectionResetError: - print('Connection reset') + for i in range(5): + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + try: + s.connect((HOST, PORT)) + s.sendall(b'hello') + data = s.recv(1024) + except ConnectionResetError: + print('Connection reset') threads = [] for i in range(500): - t = threading.Thread(target=run, args=(i,)) - threads.append(t) + t = threading.Thread(target=run, args=(i,)) + threads.append(t) for t in threads: - t.start() + t.start() for t in threads: - t.join() + t.join() ``` - [Where problems could occur] When this fix was proposed upstream, the opening line from the submitter read 'For reasons historically unknown, OpenVPN sets the listen() backlog queue to "1"'... - While changing the queue size did not cause regressions in the version of OpenVPN where the patch was applied (nor in the subsequent versions), backporting the patch to an arbitrary openVPN version could reveal a "historically unknown reason", forcing us to either fix it or revert this patch. - + While changing the queue size did not cause regressions in the version + of OpenVPN where the patch was applied (nor in the subsequent versions), + backporting the patch to an arbitrary openVPN version could reveal a + "historically unknown reason", forcing us to either fix it or revert + this patch. + + Such regressions may include networking errors, mostly related to the + TCP features of the openvpn server. One symptom that can occur when the + requests are not being processed in time is having the following message + in demesg output: "Possible SYN flooding on port ${PORT}". + [Other Info] - - Upstream bug report with relevant discussion: https://community.openvpn.net/openvpn/ticket/1208 + + Upstream bug report with relevant discussion: + https://community.openvpn.net/openvpn/ticket/1208 Upstream patch and more relevant discussion: https://www.mail- archive.com/[email protected]/msg18758.html - [Original report] See upstream bug reports: - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=968942 - https://community.openvpn.net/openvpn/ticket/1208 Openvpn < 2.4.8 opens the TCP port with a too small backlog, and on kernels > 4.3 that leads to incoming connections being dropped. This kernel message is a symptom: TCP: request_sock_TCP: Possible SYN flooding on port 1194. Dropping request. Check SNMP counters. I experienced this on a Bionic 18.04.5 (after having upgraded from Xenial) with openvpn 2.4.4-2ubuntu1.5 Fixed upstream. ** Attachment added: "reproducer_impish_focal.sh" https://bugs.launchpad.net/ubuntu/+source/openvpn/+bug/1934781/+attachment/5512611/+files/reproducer_impish_focal.sh -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1934781 Title: TCP socket backlog set too low ("request_sock_TCP: Possible SYN flooding on port ...") To manage notifications about this bug go to: https://bugs.launchpad.net/openvpn/+bug/1934781/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
