[issue1545463] New-style classes fail to cleanup attributes
Changes by Antoine Pitrou pit...@free.fr: -- assignee: belopolsky - status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Roundup Robot added the comment: New changeset 8a5bebea9fec by Antoine Pitrou in branch 'default': Issue #1545463: At shutdown, defer finalization of codec modules so that stderr remains usable. http://hg.python.org/cpython/rev/8a5bebea9fec -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Richard Oudkerk added the comment: The test seems to be failing on Windows. -- nosy: +sbt status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: The test seems to be failing on Windows. Yes. I'll try to setup a new Windows dev environment and take a look :-/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Richard Oudkerk added the comment: I think the problem is that the __del__ method fails on Windows, maybe because sys.stdout and sys.__stderr__ have been replaced by None. Consider the following program: import os class C: def __del__(self, write=os.write): write(1, bBEFORE\n) print(__del__ called) write(1, bAFTER\n) l = [C()] l.append(l) On Unix I get BEFORE __del__ called AFTER but on Windows I only get BEFORE I would suggest using os.write() instead of print() in the tests. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Richard Oudkerk added the comment: I will try a fix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Richard Oudkerk added the comment: On Windows my encoding for stdout, stderr is cp1252 which is implemented in pure python. By the time that _PyGC_DumpShutdownStats() runs the encoding.cp1252 module has been purged so stdout and stderr are broken. I am afraid I will have to leave this to you Antoine... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: Your diagnosis seems right about test_garbage_at_shudown (I can reproduce under Linux using `PYTHONIOENCODING=iso8859-15 ./python -m test -v test_gc`). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: Here is a patch, it seems to work on the custom buildbots. The problem was two-fold: - PyErr_Warn() is too high-level, it will invoke linecache and others - encodings and codecs shouldn't be cleared before the final shutdown -- Added file: http://bugs.python.org/file30172/better_shutdown.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Roundup Robot added the comment: New changeset f0833e6ff2d2 by Antoine Pitrou in branch 'default': Issue #1545463: Global variables caught in reference cycles are now garbage-collected at shutdown. http://hg.python.org/cpython/rev/f0833e6ff2d2 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: This issue is endly fixed in 3.4. Since changing the shutdown sequence is a delicate change, I won't backport to bugfix branches. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Changes by Armin Rigo ar...@users.sourceforge.net: -- nosy: -arigo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: Here is an updated patch after the latest changes on default. -- Added file: http://bugs.python.org/file30129/gcshutdown2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Antoine Pitrou added the comment: Here is a patch adding a call to gc.collect() after cleaning up most modules, with tests. -- keywords: +patch nosy: +ncoghlan stage: commit review - patch review type: resource usage - enhancement versions: +Python 3.4 -Python 3.2 Added file: http://bugs.python.org/file30126/gcshutdown.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Changes by Brett Cannon br...@python.org: -- nosy: -brett.cannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: Does anyone want to weigh in on this? I am merging in the issue812369 nosy list. I would like to either apply gc-import.patch or close this as superseded by issue812369. -- keywords: +needs review -patch nosy: +arigo, brett.cannon, candrea, cburroughs, christian.heimes, glchapman, gregory.p.smith, loewis, nascheme, pitrou -BreamoreBoy stage: - commit review versions: +Python 3.2 -Python 2.5, Python 2.6, Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Gregory P. Smith g...@krypto.org added the comment: looks harmless to me. though i think issue812369 looks okay as well at first glance. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Changes by Facundo Batista facu...@taniquetil.com.ar: -- versions: -Python 2.5.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Mark Lawrence breamore...@yahoo.co.uk added the comment: Reopened because the history shows comments and patches months after it was set to closed and won't fix, see msg61886. -- nosy: +BreamoreBoy resolution: wont fix - status: closed - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: This is superseded by issue812369, but as a stop-gap measure, I don't see any downside of applying gc-import.patch. -- assignee: - belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Alexander Belopolsky [EMAIL PROTECTED] added the comment:
amaury What if PyGC_Collect() is called just before?
That would work. With the following patch:
===
--- Python/import.c (revision 67183)
+++ Python/import.c (working copy)
@@ -498,7 +498,10 @@
PyDict_SetItem(modules, key, Py_None);
}
}
-
+ /* Collect garbage remaining after deleting the
+ modules. Mostly reference cycles created by new style
+ classes. */
+ PyGC_Collect();
/* Next, delete sys and __builtin__ (in that order) */
value = PyDict_GetItemString(modules, sys);
if (value != NULL PyModule_Check(value)) {
$ ./python.exe x.py
creating X('new')
creating X('old')
deleting X('old')
deleting X('new')
--
keywords: +patch
Added file: http://bugs.python.org/file11983/gc-import.patch
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1545463
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Changes by Alexander Belopolsky [EMAIL PROTECTED]: -- versions: +Python 2.5.3, Python 2.6, Python 2.7 ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1545463 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Amaury Forgeot d'Arc added the comment: In PyImport_Cleanup(), sys and __builtin__ are the last ones deleted. What if PyGC_Collect() is called just before? -- nosy: +amaury.forgeotdarc _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1545463 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Alexander Belopolsky added the comment:
The problem still exists in 2.5.1.
The explanations given so far are not correct. With x.py as before (see
attached):
import sys, gc, x
creating X('new')
creating X('old')
del x,sys.modules['x']
deleting X('old')
gc.collect()
deleting X('new')
6
which shows that the cycles in x module are resolvable by GC.
The problem is not that there are uncollectable objects but that GC is
ran on exit before x becomes dead.
import sys, gc, x
creating X('new')
creating X('old')
gc.set_debug(1)
sys.exit()
gc: collecting generation 2...
gc: objects in each generation: 463 2034 0
gc: done.
deleting X('old')
Looking at the comments in Py_Finalize, it looks like GvR intended to
run GC after destroying the modules, but it led to problems:
(from svn blame Python/pythonrun.c)
32278 gvanrossum
9025 guido /* Destroy all modules */
8403 guido PyImport_Cleanup();
9025 guido
32278 gvanrossum /* Collect final garbage. This disposes of
cycles created by
34776tim_one* new-style class definitions, for example.
34776tim_one* XXX This is disabled because it caused too
many problems. If
34776tim_one* XXX a __del__ or weakref callback triggers
here, Python code has
34776tim_one* XXX a hard time running, because even the sys
module has been
34776tim_one* XXX cleared out (sys.stdout is gone,
sys.excepthook is gone, etc).
34776tim_one* XXX One symptom is a sequence of information-
free messages
34776tim_one* XXX coming from threads (if a __del__ or
callback is invoked,
34776tim_one* XXX other threads can execute too, and any
exception they encounter
34776tim_one* XXX triggers a comedy of errors as subsystem
after subsystem
34776tim_one* XXX fails to find what it *expects* to find
in sys to help report
34776tim_one* XXX the exception and consequent unexpected
failures). I've also
34776tim_one* XXX seen segfaults then, after adding print
statements to the
34776tim_one* XXX Python code getting called.
34776tim_one*/
34776tim_one #if 0
32278 gvanrossum PyGC_Collect();
34776tim_one #endif
Commenting out PyGC_Collect() seems like a too radical solution because
no module referenced cycles get collected, not even those without
__del__.
I have not tried it yet, but it looks like a possible solution is to
call PyGC_Collect() at the end of _PyModule_Clear.
--
type: - resource usage
versions: +Python 2.5 -Python 2.4
Added file: http://bugs.python.org/file9338/x.py
_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1545463
_
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1545463] New-style classes fail to cleanup attributes
Georg Brandl added the comment: In the light of no further results, closing this bug. -- nosy: +georg.brandl resolution: - wont fix status: open - closed _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1545463 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
