[issue17399] test_multiprocessing hang on Windows, non-sockets
brian yardy added the comment: All 4 or 5 times I tried on 3.2, yes. In Command Prompt, 3.2 gave same error as before, 3.3 a different error. multi-test.txt has full tracebacks.'http://www.einstantloan.co.uk/' -- nosy: +brianyardy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: This is no longer a 3.x issue. 3.2 and 3.3 get security fixes only. For 3.4, test_multiprocessing is split into 4 files and all run in multiple tries. Test_multiprocessing_spawn takes a minute, but it does 264 + 20 skipped tests, including a few 'wait' tests. Three tries with 2.7 also passed. -- resolution: - out of date stage: needs patch - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: running kill-python results in Warning -- threading._dangling was modified by test_multiprocessing Warning -- multiprocessing.process._dangling was modified by test_multiprocessing test test_multiprocessing failed -- multiple errors occurred; run in verbose mode for details -v added the attached details, and the test finished, though failed. -- Added file: http://bugs.python.org/file29385/test_multi.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Richard Oudkerk added the comment: Does this happen every time you run the tests? (I don't see these errors.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: All 4 or 5 times I tried on 3.2, yes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Richard Oudkerk added the comment:
Could you try the following program:
import socket
import multiprocessing
import multiprocessing.reduction
import multiprocessing.connection
def socketpair():
with socket.socket() as l:
l.bind(('localhost', 0))
l.listen(1)
s = socket.socket()
s.connect(l.getsockname())
a, _ = l.accept()
return s, a
def bar(s):
print(s)
s.sendall(b'from bar')
if __name__ == '__main__':
a, b = socketpair()
p = multiprocessing.Process(target=bar, args=(b,))
p.start()
b.close()
print(a.recv(100))
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17399
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: In Command Prompt, 3.2 gave same error as before, 3.3 a different error. multi-test.txt has full tracebacks. -- Added file: http://bugs.python.org/file29388/mult-test.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Richard Oudkerk added the comment: Now could you try the attached file? (It will not work on 2.7 because a missing socket.fromfd().) P.S. It looks like the error for 3.3 is associated with a file f:\python\mypy\traceback.py which presumably clashes with the one in the standard library. -- Added file: http://bugs.python.org/file29389/inherit_socket.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: According to difflib, the attached file is identical with the code in your previous message. With my traceback renamed, both files print b'from bar' with 3.3 and the same error message as before with 3.2. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: Let me try downloading again. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: Both 3.2 and 3.3 give essentially the same traceback as 3.2 did before, both with installed python and yesterdays debug builds. -- Added file: http://bugs.python.org/file29391/multi-test2.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Richard Oudkerk added the comment: Both 3.2 and 3.3 give essentially the same traceback as 3.2 did before, both with installed python and yesterdays debug builds. It looks like on your machine socket handles are not correctly inherited by child processes -- I had assumed that they always would be. I suppose to fix things for 3.2 and earlier it would be necessary to backport the functionality of socket.socket.share() and socket.fromshare() from 3.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Changes by Terry J. Reedy tjre...@udel.edu: -- Removed message: http://bugs.python.org/msg184042 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Changes by Terry J. Reedy tjre...@udel.edu: -- Removed message: http://bugs.python.org/msg184043 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: My original report was for 32 bit debug build on 64 bit Win 7 machine. I just re-ran test_multiprocessing with installed 64 bit python with same result. Was I don't see these errors. on different Windows or non-Windows. One option is to skip the failing sub-tests on Windows, like some other sub-tests: skipped 'does not work with windows sockets' and consider the limitation on use of multi-processing in 2.7,3.2 as won't fix. (My view is that 3.2 users should upgrade as soon as dependencies allow.) Backporting new features requires pydev discussion. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Richard Oudkerk added the comment: My original report was for 32 bit debug build on 64 bit Win 7 machine. I just re-ran test_multiprocessing with installed 64 bit python with same result. Was I don't see these errors. on different Windows or non-Windows. On 64-bit Windows 7 with both 32 and 64 bit builds. One option is to skip the failing sub-tests on Windows, like some other sub-tests: skipped 'does not work with windows sockets' and consider the limitation on use of multi-processing in 2.7,3.2 as won't fix. (My view is that 3.2 users should upgrade as soon as dependencies allow.) Yes, I would be inclined to do that. Backporting new features requires pydev discussion. I only meant exposing that functionality in the private _multiprocessing extension. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: Same OS, different hardware, drivers, and buried settings, surprisingly different behavior. Oh, well. I will test any changes you decide on that need testing. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
New submission from Terry J. Reedy:
test_multiprocessing is giving multiple errors like
Process Process-21:
Traceback (most recent call last):
File F:\Python\dev\py27\lib\multiprocessing\process.py, line 258, in
_bootstrap
self.run()
File F:\Python\dev\py27\lib\multiprocessing\process.py, line 114, in run
self._target(*self._args, **self._kwargs)
File F:\Python\dev\py27\lib\test\test_multiprocessing.py, line 1703, in
_test
conn.send('hello')
IOError: [Errno 10038] An operation was attempted on something that is not a
socket
...
File F:\Python\dev\py27\lib\test\test_multiprocessing.py, line 1419, in
_putter
manager.connect()
File F:\Python\dev\py27\lib\multiprocessing\managers.py, line 500, in
connect
conn = Client(self._address, authkey=self._authkey)
File F:\Python\dev\py27\lib\multiprocessing\connection.py, line 465, in
XmlClient
return ConnectionWrapper(Client(*args, **kwds), _xml_dumps, _xml_loads)
File F:\Python\dev\py27\lib\multiprocessing\connection.py, line 175, in
Client
answer_challenge(c, authkey)
File F:\Python\dev\py27\lib\multiprocessing\connection.py, line 420, in
answer_challenge
message = connection.recv_bytes(256) # reject large message
IOError: [Errno 10038] An operation was attempted on something that is not a
socket
After a few of these, it hangs and I have to either kill multiple processes
with TaskManager (or maybe kill-python would work) or kill the console. All
tracebacks seem to one of these two. The common factor is 'not a socket'. It
seems that something in the test should be skipped on windows. I got similar
results running test_multiprocessing by itself. The process numbers are
haphazard.
--
messages: 184000
nosy: ezio.melotti, sbt, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: test_multiprocessing hang on Windows, non-sockets
type: behavior
versions: Python 2.7, Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17399
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17399] test_multiprocessing hang on Windows, non-sockets
Terry J. Reedy added the comment: For 3.3 and 3.4, test went on for several minutes putting out [## refs] lines but finally said '1 test OK'. I tried 3.2 again and it never outputs refs lines but fairly soon gives errors. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
