[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
darrenr python-roun...@dranalli.com added the comment: Regardless of the type of algorithm used by the garbage collector that currently exists in Python, its worst-case performance is undesirable. I have some interest in implementing a garbage collector for Python with improved performance characteristics but don't really have the time for it. Hopefully someone does. In any case thanks for hearing me out. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
darrenr python-roun...@dranalli.com added the comment: I ask because in my opinion a three-second pause on a modern machine is significant for any program with any sort of interactivity--significant enough to warrant a warning in the documentation. Python is a great language and I think it deserves an incremental implementation of garbage collection. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Martin v. Löwis mar...@v.loewis.de added the comment: Python is a great language and I think it deserves an incremental implementation of garbage collection. Python's cyclic garbage collector is incremental. If you can provide a specific patch to replace it with something better, please submit it as a separate issue to the tracker. If you can hire somebody to implement such a thing, go right ahead. Otherwise, I think there is little that can be done. Python gets all of its contributions from volunteers. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Martin v. Löwis mar...@v.loewis.de added the comment: Hard real-time applications written in Python should not rely on the cyclic garbage collector. They should call gc.disable at startup, and completely rely on reference counting for releasing memory. Doing so might require rewrites to the application, explicitly breaking cycles so that reference counting can release them. Even with cyclic gc disabled, applications worried about meeting hard deadlines need to look even more into memory allocation and deallocation; e.g. releasing a single object may cause a chained release of many objects, which can affect worst case execution times. There are more issues to consider (which are all out of scope of the bug tracker). -- nosy: +loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
darrenr python-roun...@dranalli.com added the comment: OK cool, that's the development strategy we've already adopted. Is this limitation of Python's garbage collector in relation to real-time applications documented anywhere? ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Martin v. Löwis mar...@v.loewis.de added the comment: OK cool, that's the development strategy we've already adopted. Is this limitation of Python's garbage collector in relation to real-time applications documented anywhere? Why do you ask? (this is OT for the bug tracker) It's not in the Python documentation. However, any good book on real-time systems will tell you that garbage collection is a problem. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
New submission from darrenr python-roun...@dranalli.com: Python's garbage collector holds GIL during collection and doesn't provide any method of interruption or concurrency with other Python threads within a single Python VM. This can be a problem for realtime applications. The worst-case performance of the garbage collector takes linear time with respect to the number of Python objects that could potentially be involved in a garbage cycle. I've attached timings taken on a Core 2 Quad 2.4 GHz (WinXP Pro, 3GB RAM), with ever-increasing numbers of objects. The gc at worst takes upwards of 3 seconds before the process runs out of memory. If gc periodically released the GIL it would allow it to be put in a separate thread, but as it stands it blocks the Python VM for periods of time that are too long for realtime interactive applications. Alternatively a gc that is incremental by default would eliminate the need for a second thread. -- files: gctimings.zip messages: 78646 nosy: darrenr severity: normal status: open title: garbage collector blocks and takes worst-case linear time wrt number of objects versions: Python 2.4, Python 2.6, Python 3.0 Added file: http://bugs.python.org/file12510/gctimings.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Changes by darrenr python-roun...@dranalli.com: -- components: +Interpreter Core type: - resource usage ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Changes by David W. Lambert lamber...@corning.com: -- nosy: +LambertDW ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
Benjamin Peterson benja...@python.org added the comment: The garbage collector will never be able to run in a second thread because it manipulates Python objects, which the GIL is supposed to protect! As for non-linear complexity, see #4688 and #4074 for some attempts to sooth this problem over. -- nosy: +benjamin.peterson resolution: - duplicate status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4794] garbage collector blocks and takes worst-case linear time wrt number of objects
darrenr python-roun...@dranalli.com added the comment: A 'stop-the-world' garbage collector that periodically released the GIL could be run in a second thread, allowing the main thread to break in and do some processing. However the nature of a stop-the-world collector means that it probably would not easily be able to deal with changes made by other threads in the middle of the collect. My concern is that the Python process blocks and is unresponsive due to garbage collection for periods of time that are not acceptable for realtime interactive applications. Are there any plans to add an incremental collector to Python? ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4794 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
