Re: [PyQt] subtle bug in PyQt in combination with Python garbage collector
On Thu, 25 Aug 2011 18:08:17 +0100, Phil Thompson
p...@riverbankcomputing.com wrote:
On Sat, 13 Aug 2011 10:05:14 -0600, Kovid Goyal ko...@kovidgoyal.net
wrote:
This bug has been present for a very long time. As a workaround in my
projects,
I disable the automatic garbage collector and run garbage collection
manually
in the GUI thread via QTimer. Here's the code to do that:
class GarbageCollector(QObject):
'''
Disable automatic garbage collection and instead collect manually
every INTERVAL milliseconds.
This is done to ensure that garbage collection only happens in the
GUI
thread, as otherwise Qt can crash.
'''
INTERVAL = 5000
def __init__(self, parent, debug=False):
QObject.__init__(self, parent)
self.debug = debug
self.timer = QTimer(self)
self.timer.timeout.connect(self.check)
self.threshold = gc.get_threshold()
gc.disable()
self.timer.start(self.INTERVAL)
#gc.set_debug(gc.DEBUG_SAVEALL)
def check(self):
#return self.debug_cycles()
l0, l1, l2 = gc.get_count()
if self.debug:
print ('gc_check called:', l0, l1, l2)
if l0 self.threshold[0]:
num = gc.collect(0)
if self.debug:
print ('collecting gen 0, found:', num, 'unreachable')
if l1 self.threshold[1]:
num = gc.collect(1)
if self.debug:
print ('collecting gen 1, found:', num,
'unreachable')
if l2 self.threshold[2]:
num = gc.collect(2)
if self.debug:
print ('collecting gen 2, found:', num,
'unreachable')
def debug_cycles(self):
gc.collect()
for obj in gc.garbage:
print (obj, repr(obj), type(obj))
Kovid.
Thanks for the insight.
Two solutions spring to mind...
1. Implement the above so that it is created when QThread.start() is
called for the first time.
2. Change the SIP generated dtor code so that it calls deleteLater() if
the object's thread is not the current one...
if (QThread::currentThread() == obj-thread())
delete obj;
else
obj-deleteLater();
I'd prefer the latter (assuming it works).
Thoughts?
Tonight's SIP snapshot will implement the second solution. Let me know the
results.
Phil
___
PyQt mailing listPyQt@riverbankcomputing.com
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Re: [PyQt] subtle bug in PyQt in combination with Python garbage collector
On Sat, 13 Aug 2011 10:05:14 -0600, Kovid Goyal ko...@kovidgoyal.net
wrote:
This bug has been present for a very long time. As a workaround in my
projects,
I disable the automatic garbage collector and run garbage collection
manually
in the GUI thread via QTimer. Here's the code to do that:
class GarbageCollector(QObject):
'''
Disable automatic garbage collection and instead collect manually
every INTERVAL milliseconds.
This is done to ensure that garbage collection only happens in the
GUI
thread, as otherwise Qt can crash.
'''
INTERVAL = 5000
def __init__(self, parent, debug=False):
QObject.__init__(self, parent)
self.debug = debug
self.timer = QTimer(self)
self.timer.timeout.connect(self.check)
self.threshold = gc.get_threshold()
gc.disable()
self.timer.start(self.INTERVAL)
#gc.set_debug(gc.DEBUG_SAVEALL)
def check(self):
#return self.debug_cycles()
l0, l1, l2 = gc.get_count()
if self.debug:
print ('gc_check called:', l0, l1, l2)
if l0 self.threshold[0]:
num = gc.collect(0)
if self.debug:
print ('collecting gen 0, found:', num, 'unreachable')
if l1 self.threshold[1]:
num = gc.collect(1)
if self.debug:
print ('collecting gen 1, found:', num,
'unreachable')
if l2 self.threshold[2]:
num = gc.collect(2)
if self.debug:
print ('collecting gen 2, found:', num,
'unreachable')
def debug_cycles(self):
gc.collect()
for obj in gc.garbage:
print (obj, repr(obj), type(obj))
Kovid.
Thanks for the insight.
Two solutions spring to mind...
1. Implement the above so that it is created when QThread.start() is
called for the first time.
2. Change the SIP generated dtor code so that it calls deleteLater() if
the object's thread is not the current one...
if (QThread::currentThread() == obj-thread())
delete obj;
else
obj-deleteLater();
I'd prefer the latter (assuming it works).
Thoughts?
Phil
___
PyQt mailing listPyQt@riverbankcomputing.com
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Re: [PyQt] subtle bug in PyQt in combination with Python garbage collector
On 2011-08-13 18:05, Kovid Goyal wrote: This bug has been present for a very long time. As a workaround in my projects, I disable the automatic garbage collector and run garbage collection manually in the GUI thread via QTimer. Here's the code to do that: class GarbageCollector(QObject): I just wanted to say thank you for sharing this info. I've applied your workaround on TortoiseHg [1] https://bitbucket.org/tortoisehg/thg/changeset/d5a9ae16b56b which will be included in the next stable release 2.1.3 (due on Sep 1st). [1] http://tortoisehg.bitbucket.org/ ___ PyQt mailing listPyQt@riverbankcomputing.com http://www.riverbankcomputing.com/mailman/listinfo/pyqt
Re: [PyQt] subtle bug in PyQt in combination with Python garbage collector
This bug has been present for a very long time. As a workaround in my projects,
I disable the automatic garbage collector and run garbage collection manually
in the GUI thread via QTimer. Here's the code to do that:
class GarbageCollector(QObject):
'''
Disable automatic garbage collection and instead collect manually
every INTERVAL milliseconds.
This is done to ensure that garbage collection only happens in the GUI
thread, as otherwise Qt can crash.
'''
INTERVAL = 5000
def __init__(self, parent, debug=False):
QObject.__init__(self, parent)
self.debug = debug
self.timer = QTimer(self)
self.timer.timeout.connect(self.check)
self.threshold = gc.get_threshold()
gc.disable()
self.timer.start(self.INTERVAL)
#gc.set_debug(gc.DEBUG_SAVEALL)
def check(self):
#return self.debug_cycles()
l0, l1, l2 = gc.get_count()
if self.debug:
print ('gc_check called:', l0, l1, l2)
if l0 self.threshold[0]:
num = gc.collect(0)
if self.debug:
print ('collecting gen 0, found:', num, 'unreachable')
if l1 self.threshold[1]:
num = gc.collect(1)
if self.debug:
print ('collecting gen 1, found:', num, 'unreachable')
if l2 self.threshold[2]:
num = gc.collect(2)
if self.debug:
print ('collecting gen 2, found:', num, 'unreachable')
def debug_cycles(self):
gc.collect()
for obj in gc.garbage:
print (obj, repr(obj), type(obj))
Kovid.
On Sat, Aug 13, 2011 at 01:11:38PM +0200, Erik Janssens wrote:
Hello Phil,
I believe to have found a subtle bug in PyQt in
combination with the Python garbage collector.
Here is what I think that happens :
- A QObject is constructed in one thread,
and this QObject construction contains a cyclic
dependency.
- this construction will not be deleted with
reference counting, but only when the garbage
collector is triggered
- now, if the garbage collector starts collecting
in a different thread then the one that the
QObject was constructed in, the QObject gets
destroyed in that different thread
- when a QObject is destroyed, it sends events to
its parent object. these events are then send in
the wrong thread
- this leads to corruption and/or segmentation faults
To see this happening, please run the attached test
case with a development build of Qt (because then,
you see the assertion failure).
I have observed this with :
- Qt 4.7.2
- PyQt 4.8.3
- sip 4.12.1
PySide suffers from the same behavior.
Thank you and best regards,
Erik
!DSPAM:3,4e465c069887238618555!
Test the behaviour of the qt bindings in various circumstances.
import unittest
from PyQt4 import QtGui, QtCore
class GarbageCollectionCase( unittest.TestCase ):
def setUp(self):
self.application = QtGui.QApplication.instance()
if not self.application:
import sys
self.application = QtGui.QApplication(sys.argv)
def test_cyclic_dependency( self ):
Create 2 widgets with a cyclic dependency, so that they can
only be removed by the garbage collector, and then invoke the
garbage collector in a different thread.
import gc
class CyclicChildWidget(QtGui.QWidget):
def __init__( self, parent ):
super( CyclicChildWidget, self ).__init__( parent )
self._parent = parent
class CyclicWidget(QtGui.QWidget):
def __init__( self ):
super( CyclicWidget, self ).__init__()
CyclicChildWidget( self )
# turn off automatic garbage collection, to be able to trigger it
# at the 'right' time
gc.disable()
alive = lambda :sum( isinstance(o,CyclicWidget) for o in
gc.get_objects() )
#
# first proof that the wizard is only destructed by the garbage
# collector
#
cycle = CyclicWidget()
self.assertTrue( alive() )
del cycle
self.assertTrue( alive() )
gc.collect()
self.assertFalse( alive() )
#
# now run the garbage collector in a different thread
#
cycle = CyclicWidget()
del cycle
self.assertTrue( alive() )
class GarbageCollectingThread(QtCore.QThread):
def run(thread):
self.assertTrue( alive() )
# assertian failure here, and core dump
gc.collect()
self.assertFalse( alive() )
thread = GarbageCollectingThread()
