On Wed, 2011-02-09 at 10:57 +0100, Martin Dobias wrote: > Hi list, > > for Quantum GIS project [1] we have been adding support for rendering > of maps in threads by the means of QtConcurrent framework. Rendering > is started asynchronously from the main thread, spawning several > worker threads where all the painting is done. This works very well. > > We have also PyQt4-based bindings which are used by 3rd party plugins. > The rendering stack allows custom implementation of some classes. The > problem starts at the point if any part of the rendering stack (now > running in worker threads) is implemented in Python. Please note that > this is not so typical situation since the core implementation is in > C++, this is just the case when a Python plugin implements custom > rendering for one of the map layers. > > With the multithreaded rendering and Python code in the rendering > stack, the rendering gets stuck: GIL that has been acquired by the > main thread has not been released, so the Python code in a worker > thread keeps waiting for GIL. My question is if there is any simple > and elegant solution for this problem. > > First thing I have tried was to run some Python code in the main > thread while multithreaded rendering is waiting for GIL. This worked > as expected: Python interpreter periodically released and acquired > GIL, so the worker thread was eventually able to finish its work. So a > silly solution would be to use a timer to run a small amount of Python > code in intervals in order to allow worker thread to acquire GIL. I > know that is stupid and ineffective. > > I have also tried to explicitly release GIL when rendering starts and > acquire it again when finished. This worked, but only until I have > tried to run some Python code in main thread while still rendering. I > understand that trying to run Python code without acquiring GIL will > inevitably lead to a segfault, though I have failed to find a way > around this with Python API (the docs are clear as mud for this topic, > as someone already noted). Ideally I would like to tell SIP that when > it is going to run some Python code (e.g. when a slot has been > invoked), it has to acquire GIL, run it, and finally release GIL again > - so that rendering in threads can continue. > > My last idea was that if I were able to identify whether any class of > the rendering stack (for a map layer) has Python implementation, I > could run the rendering of such map layer in main thread, effectively > avoiding the multithreading issues. But I would like to solve it more > generically with correct GIL handling. > > Please do you have any ideas how to solve this problem?
Which version of PyQt4/sip are you using? In recent PyQt versions, GIL is always released when entering Qt. In fact, sip does not release the GIL by default when calling a C++ libraries, but it supports a "-g" option that does just that. You can check that PyQt4's configure.py, in sip_set_flags(), passes "-g" to sip while compiling. You might want to make sure that all your bindings/plugins/whatever that are sip-based are compiled with the "-g" flag. -- Giovanni Bajo :: [email protected] Develer S.r.l. :: http://www.develer.com My Blog: http://giovanni.bajo.it Last post: Compile-time Function Execution in D _______________________________________________ PyQt mailing list [email protected] http://www.riverbankcomputing.com/mailman/listinfo/pyqt
