[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Roundup Robot added the comment: New changeset 1794d754ff3c by Steve Dower in branch 'default': Issue #23060: Suppresses a multiprocessing assert that fails incorrectly https://hg.python.org/cpython/rev/1794d754ff3c -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Steve Dower added the comment: I've commented out the assertion for now with a comment pointing to this issue, so that'll keep the buildbots running while we figure out how to deal with this properly. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Tim Golden added the comment: I agree that this is a fragile assertion; it's too far removed from the CreateFileMapping call which can generate it and almost impossible to work around (in calling code) if it should fail in the way we're seeing in the buildbot. I think we're better off relying on a genuine exception bubbling up from the CreateFileMapping/MapViewOfFile calls than trying to assert the no-exception error return. While the preallocate self.buffer hack you mention would probably have the effect of preventing the assertion, it's really just adding another layer of unwanted complexity. (And might still not work in some future memory-allocation algorithm). @Richard: if you're watching, have I missed anything? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Antoine Pitrou added the comment: Or perhaps: buffer = mmap.mmap(-1, self.size, tagname=self.name) assert _winapi.GetLastError() == _winapi.ERROR_ALREADY_EXISTS self.buffer = buffer ? -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Steve Dower added the comment: That was actually my first hack and it also works. The fundamental problem here is that GetLastError() is not actually meant to indicate *whether* an error has occurred, but when one has occurred it describes what it was. Indeed, in this case no error has occurred - CreateFileMapping was called and there is a file mapping available. OpenFileMapping is provided if an error is required when the mapping does not already exist. The best option is definitely a separate method (or parameter) on mmap to open/raise rather than open/create. I may just comment out the assertion with a reference to this bug in the meantime, so that the buildbots aren't getting stuck every time. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
New submission from Steve Dower: Currently, the implementation of __setstate__ at Lib/multiprocessing/heap.py:54 looks like this: def __setstate__(self, state): self.size, self.name = self._state = state self.buffer = mmap.mmap(-1, self.size, tagname=self.name) assert _winapi.GetLastError() == _winapi.ERROR_ALREADY_EXISTS This assertion can fail when the assignment to buffer triggers memory [re]allocation, which could clear the last error result. So far, this fails reliably on debug builds using VS 2015 (which causes some tests to timeout when all of the child processes fail to start), but could also fail in threaded code at any time. I don't know why release builds or builds with VS 2010 did not trigger this, but I would speculate that either VS 2015 is using a different API to allocate memory or Windows 8.1 has changed the behaviour of an API. Whether a function resets the last error code is deliberately unspecified (http://msdn.microsoft.com/en-us/library/windows/desktop/ms679360(v=vs.85).aspx - some functions set the last-error code to 0 on success and others do not). In my opinion, the assertion should just be removed. A hack that appears to work currently is to add self.buffer = None before the existing assignment (to pre-expand self.__dict__ and avoid the allocation). If the assertion is important, someone may want to add a parameter to mmap() to require that the memory-mapped file already exists and throws otherwise, but I am not volunteering to do this. -- components: Windows messages: 232697 nosy: jnoller, sbt, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows versions: Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23060] Assert fails in multiprocessing.heap.Arena.__setstate__ on Windows
Steve Dower added the comment: A buildbot failure due to this can be seen here: http://buildbot.python.org/all/builders/AMD64%20Windows8%203.x/builds/183/steps/test/logs/stdio -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23060 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
