[issue38697] test.support.find_unused_port() race condition: test_socket fails with OSError: [Errno 98] Address already in use

[STINNER Victor]

STINNER Victor  added the comment:

support.find_unused_port() has a race condition by design, but tests are re-run 
sequentially by buildbots and so the rare race condition is worked around in 
practice.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38697] test.support.find_unused_port() race condition: test_socket fails with OSError: [Errno 98] Address already in use

[STINNER Victor]

New submission from STINNER Victor :

When two tests using support.find_unused_port() run in parallel, it's possible 
to get the same port number and so the first process using it wins, and the 
other one fails. Example with test_socket running in parallel of multiple many 
other tests.

Example of fix to avoid support.find_unused_port():

diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
index 1bf562a03d..4479a69943 100644
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -6202,10 +6202,11 @@ class CreateServerFunctionalTest(unittest.TestCase):
  "dualstack_ipv6 not supported")
 @unittest.skipUnless(support.IPV6_ENABLED, 'IPv6 required for this test')
 def test_dual_stack_client_v6(self):
-port = support.find_unused_port()
-with socket.create_server(("", port), family=socket.AF_INET6,
+with socket.create_server(("", 0), family=socket.AF_INET6,
   dualstack_ipv6=True) as sock:
 self.echo_server(sock)
+port = sock.getsockname()[1]
+print("PORT", port)
 self.echo_client(("::1", port), socket.AF_INET6)


AMD64 RHEL7 Refleaks 3.8:
https://buildbot.python.org/all/#/builders/299/builds/48

0:02:32 load avg: 7.37 [132/423] test_timeout passed (56.5 sec) -- running: 
test_regrtest (1 min 56 sec), test_logging (1 min 17 sec), test_mailbox (1 min 
39 sec), test_statistics (33.8 sec), test_socket (55.8 sec)
...
0:02:56 load avg: 8.33 [153/423] test_urllib2net passed -- running: 
test_regrtest (2 min 20 sec), test_logging (1 min 40 sec), test_mailbox (2 min 
2 sec), test_statistics (57.4 sec), test_socket (1 min 19 sec)
...
0:03:11 load avg: 8.30 [161/423] test_asyncore passed -- running: test_logging 
(1 min 55 sec), test_support (33.7 sec), test_mailbox (2 min 17 sec), 
test_statistics (1 min 12 sec), test_socket (1 min 34 sec)
...
0:03:24 load avg: 8.01 [167/423] test_support passed (46.7 sec) -- running: 
test_mailbox (2 min 31 sec), test_multiprocessing_forkserver (40.2 sec), 
test_socket (1 min 47 sec)
...
0:03:36 load avg: 7.78 [180/423] test_docxmlrpc passed -- running: test_decimal 
(30.7 sec), test_mailbox (2 min 42 sec), test_multiprocessing_forkserver (52.1 
sec), test_socket (1 min 59 sec)
...
0:03:39 load avg: 7.88 [184/423/1] test_socket failed (2 min 2 sec) -- running: 
test_decimal (33.9 sec), test_mailbox (2 min 46 sec), 
test_multiprocessing_forkserver (55.2 sec)
...
0:03:39 load avg: 7.88 [184/423/1] test_socket failed (2 min 2 sec) -- running: 
test_decimal (33.9 sec), test_mailbox (2 min 46 sec), 
test_multiprocessing_forkserver (55.2 sec)
beginning 6 repetitions
123456


Warning -- threading._dangling was modified by test_socket
  Before: {}
  After:  {, , , } 

test test_socket failed -- Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.8.cstratak-RHEL7-x86_64.refleak/build/Lib/test/test_socket.py",
 line 6206, in test_dual_stack_client_v6
with socket.create_server(("", port), family=socket.AF_INET6,
  File 
"/home/buildbot/buildarea/3.8.cstratak-RHEL7-x86_64.refleak/build/Lib/socket.py",
 line 886, in create_server
raise error(err.errno, msg) from None
OSError: [Errno 98] Address already in use (while attempting to bind on address 
('', 45626))

--
components: Tests
messages: 356022
nosy: vstinner
priority: normal
severity: normal
status: open
title: test.support.find_unused_port() race condition: test_socket fails with 
OSError: [Errno 98] Address already in use
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com