[issue19021] AttributeError in Popen.__del__
Larry Hastings added the comment: Those six revisions have been cherry-picked into 3.4.0. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- stage: needs patch - committed/rejected ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Larry Hastings added the comment: So you're asking that I cherry pick six revisions here? 6a1711c96fa6 fa160c8145e5 efaf12106d68 7ecee9e0dc58 10ea3125d7b8 488ccbee6ee6 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Changes by Barry A. Warsaw ba...@python.org: -- priority: normal - release blocker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: I've testing this patch on Ubuntu, and it seems to fix the problem. My quick testing doesn't show any new problems, but we'll only know for sure once the new Python 3.4 package hits the archive and folks start updating to it. So far so good though. Larry, please consider cherry picking this into 3.4.0 final. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: Nosying Doko, since I think he may want to get this fix into Debian, if Larry does not cherry pick it into 3.4.0 final. -- nosy: +doko ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Serhiy Storchaka added the comment: Note that the changeset 6a1711c96fa6 introdused several errors (mostly tests failures), so it needs other changesets to fix it. Related changesets which should be cherry picked: 6a1711c96fa6, fa160c8145e5, efaf12106d68, 7ecee9e0dc58, 10ea3125d7b8, 488ccbee6ee6. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: On Feb 28, 2014, at 09:42 PM, Serhiy Storchaka wrote: Note that the changeset 6a1711c96fa6 introdused several errors (mostly tests failures), so it needs other changesets to fix it. Related changesets which should be cherry picked: 6a1711c96fa6, fa160c8145e5, efaf12106d68, 7ecee9e0dc58, 10ea3125d7b8, 488ccbee6ee6. Interestingly enough, the test failures don't seem to break the build, but do seem to break the DEP 8 tests, which are run on the built package and must pass before the package is promoted to the primary archive. It takes quite a while to build Python for Ubuntu (since we do several builds, e.g. debug builds, etc. along with all the tests), so I've just noticed this. Thanks for listing the additional revisions that need to be cherry picked. I will investigate for Ubuntu, but it does convince me more that Larry should attempt to pull these into 3.4.0. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: We're seeing this in Ubuntu now that 3.4 is the default. https://bugs.launchpad.net/python/+bug/1284469 -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
STINNER Victor added the comment: Serhiy fixed the issue #19255 with the changeset 6a1711c96fa6, but this changeset will wait Python 3.4.1. -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: On Feb 27, 2014, at 03:46 PM, STINNER Victor wrote: Serhiy fixed the issue #19255 with the changeset 6a1711c96fa6, but this changeset will wait Python 3.4.1. Okay, thanks. I was reviewing and rather liked the less invasive patch, but if this one is going to make it into 3.4.1, I'll review it and test it locally. We can patch Ubuntu's 3.4 for now. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: If this fixes the problem, shouldn't the issue be closed and a NEWS item added? I'm going to test the patch locally. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Antoine Pitrou added the comment: Hmm... if this *already* affects Ubuntu users, shouldn't this be fixed in 3.4 proper? It's extremely likely that someone else will be affected too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Barry A. Warsaw added the comment: On Feb 27, 2014, at 07:23 PM, Antoine Pitrou wrote: Hmm... if this *already* affects Ubuntu users, shouldn't this be fixed in 3.4 proper? It's extremely likely that someone else will be affected too. Agreed! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Serhiy Storchaka added the comment: Here is a patch which implements Richard's suggestion: _active now contains pid-s instead of Popen instances. But this doesn't fix this issue. Patches for issue19255 and issue12085 fixes it. -- Added file: http://bugs.python.org/file34021/subprocess_active_pids.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Changes by Serhiy Storchaka storch...@gmail.com: -- stage: patch review - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Antoine Pitrou added the comment: There is a regression in 3.4 due to changes in shutdown procedure. This code correctly works in 3.3. There are more than a dozen places in the stdlib which rely upon accessibility of builtins. Well, perhaps we can special-case builtins not to be wiped at shutdown. However, there is another problem here in that the Popen object survives until the builtins module is wiped. This should be investigated too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Richard Oudkerk added the comment: Well, perhaps we can special-case builtins not to be wiped at shutdown. However, there is another problem here in that the Popen object survives until the builtins module is wiped. This should be investigated too. Maybe it is because it uses the evil resuscitate-in-__del__ trick. I presume that if the child process survives during shutdown, then the popen object is guaranteed to survive too. We could get rid of the trick: * On Windows __del__ is unneeded since we don't need to reap zombie processes. * On Unix __del__ could just add self._pid (rather than self) to the list _active. _cleanup() would then use os.waitpid() to check the pids in _active. The hardest thing about making such a change is that test_subprocess currently uses _active. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Serhiy Storchaka added the comment: There is a regression in 3.4 due to changes in shutdown procedure. This code correctly works in 3.3. There are more than a dozen places in the stdlib which rely upon accessibility of builtins. I wrote patches for all these cases, but third-party code will be broken. I think we should restore guarantees about builtins. -- versions: -Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Serhiy Storchaka added the comment: I'm wondering too. While the code in Popen.__del__() is almost same, I can't reproduce the issue in 3.3. Perhaps it relates to some 3.4 changes? Import machinery, weak references, the shutdown process? Before applying the patch which fixes Popen.__del__() I want understand what happens. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Richard Oudkerk added the comment: The clearing of modules at shutdown has been substantially changed in 3.4. Now a best effort is made to let the module go away purely by gc. Those modules which survive get purged in random order. In 3.3 all modules were purged, but builtins was special cased to be purged last. (See Python/import.c:PyImport_Cleanup().) I would favour setting a flag before the purging stage which prevents __del__ methods (and weakrefs?) from running. -- nosy: +pitrou, sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Serhiy Storchaka added the comment: Could we instead restore the builtins module to it's initial state before the purging stage? I believe all builtins are immutable and can't create reference loops. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Terry J. Reedy added the comment: Before applying the patch which fixes Popen.__del__() I think your #12085 patch should be applied and that issue closed. It is not specifically about shutdown. Tweaking shutdown further would be a new issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Terry J. Reedy added the comment: When the patch to #12085 is changed, as has been agreed, I think, this issue should go away. Moving the deletion of builtins to later in the shutdown process has be discussed and maybe implemented. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Oleg Oshmyan added the comment: But the thing is, builtins are already supposed to be the very last thing destroyed at shutdown. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Oleg Oshmyan added the comment: Anothe possible solution is get rid from getattr and catch AttributeError instead. Surely this would suffer from the same issue? Why are the builtins getting deleted anyway? In fact, why is getattr getting deleted from the builtins module? The __builtins__ global is specifically not deleted when deleting modules to let destructors use builtins, and indeed the error message says is not callable, suggesting the builtins module is still there but getattr is not. Is this perhaps some sort of IDLE bug? -- nosy: +chortos ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
New submission from Serhiy Storchaka: Sometimes closing IDLE I got such exception: Exception ignored in: bound method Popen.__del__ of subprocess.Popen object at 0xb5b8618c Traceback (most recent call last): File /home/serhiy/py/cpython/Lib/subprocess.py, line 890, in __del__ TypeError: 'NoneType' object is not callable It is happened because the builtins module was destroyed before running __del__ in process of garbage collecting. Here is a patch which should fix the bug. Anothe possible solution is get rid from getattr and catch AttributeError instead. See also issue16650. -- components: Library (Lib) files: subprocess_del_getattr.patch keywords: patch messages: 197737 nosy: serhiy.storchaka, terry.reedy priority: normal severity: normal stage: patch review status: open title: AttributeError in Popen.__del__ type: behavior versions: Python 2.7, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file31762/subprocess_del_getattr.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19021] AttributeError in Popen.__del__
Terry J. Reedy added the comment: The condition was changed from self._child_created to the getattr call in #12085. As explained in msg197748, I think that change should be reverted (and this issue closed). I think elaborating a bad patch only makes it worse. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19021 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
