On Sun, Jan 12, 2014 at 4:32 AM, iMath <[email protected]> wrote: > thanks for your great explanation ,it helped me a lot !!! > > the GIL is not present under Ironpython nor Jython and so, running > multithreaded Python code under them WILL give the expected performance > benefits . > That is correct. However, if you still want to use CPython, can also avoid the GIL with multiple processes using multiprocessing or concurrent.futures.ProcessPool. I have also had a good experience with Cython <http://cython.org/>, which of course has nothing to do with parallelism.
*A short aside:* I studied Python for high-performance computing in university and also at my internship with NCAR. My opinion: don’t use it, or move your performance-sensitive code to C/C++ extensions or Cython. There are a myriad of solutions out there that boil down to one of four things: fork(), pthreads, MPI, or CUDA. Tacking on one of [multiprocessing, multithreading, message-passing, or stream processing] to Python and expecting it to be fast is like fitting a square peg in a round hole. It’s a great academic exercise, but not very practical. That’s just my opinion, so of course take it with a grain of salt. I love Python, hence being on this list. But I don’t love it for it’s performance :) > > > http://www.neotitans.com/resources/python/python-threads-multithreading-tutorial.html > > ------------------ Original ------------------ > *From: * "Sean Fisk";<[email protected]>; > *Date: * Sun, Jan 12, 2014 02:30 PM > *To: * "iMath"<[email protected]>; > *Cc: * "pyside"<[email protected]>; > *Subject: * Re: [PySide] QtConcurrent support? > > Hello iMath, > > This is a complicated topic; one that I have been studying for the past > six months or so. I would specifically avoid the term “real > multithreading.” There are enough factors that come into play to make that > term virtually unqualified. > The GIL and Parallelism > > What it boils down to is this: > > - If Python code is running, the GIL *is in effect*. > - If Python code is not running (e.g., C, C++), the GIL *may or may > not be in effect*. > > Instead of *real multithreading*, let us instead define the term *true > in-process parallelism* as follows: > > *Given that a process has multiple instructions, true in-process > parallelism occurs when two or more instructions are executed in the same > instant.* > > Obviously, a machine would need to possess multiple processors for this to > happen. But true in-process parallelism is specifically what the GIL > prevents. Hence, no matter whether you use threading or QThread or > anything else built on these, the GIL will prevent true in-process > parallelism for pure Python code. > > However, just because two instructions can’t happen in the same instant in > Python code doesn’t make threading useless. If you want two operations to > *appear* to be executing simultaneously, threading in Python will > suffice. However, you won’t get any speed boost because your code can’t > truly execute simultaneously, thanks to the GIL. > Extension Modules > > Extension modules (e.g., written in C, C++) complicate the issue because > they have the possibility of releasing the GIL. PySide, as you know, is an > extension module. I can’t say much here, though, because AFAIK the only way > to determine whether an extension module is releasing the GIL is to look in > its source code. That’s somewhat out of my realm. > > Just know that each time you call a PySide function which eventually leads > to C++ code (or, I guess, binary which was C++ code) being executed there > is a possibility of releasing the GIL and achieving true in-process > parallelism. > QtConcurrent > > As QtConcurrent is based on QThread, using it for a speedup [if it was in > PySide] would be basically moot. That’s because all of QtConcurrent’s > hypothetical functions take a callable as an argument, which, if written in > Python, would invoke the GIL. > > In Python 2’s case, the much better options are to use the standard > multiprocessing <http://docs.python.org/2/library/multiprocessing.html>module > and > concurrent.futures <http://pythonhosted.org/futures/> (which has been > added to the standard library in Python 3.2). Each of them provide a thread > and process pool approach. As implied earlier, only the process-based > approach will allow for the possibility of speedup (when in pure Python). > These integrate much nicer with Python and are probably better than what Qt > could provide. > Bottom Line > > What you choose depends on your use case. By the way, what *are* you > trying to achieve? > > I don’t claim to be a threading or Python or GIL expert, but this is where > my research on the topic has pointed me. Please, someone correct me if you > take issue. > > Sincerely, > > > > -- > Sean Fisk > > > On Sat, Jan 11, 2014 at 11:09 PM, iMath <[email protected]> wrote: > >> In these 2 posts >> http://nathanhorne.com/?p=353 >> http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html >> >> says >> "when the threaded code is calling long running c/c++ functions, such as >> a database select, it will release the GIL until the call completes. This >> is all done automatically within python and PySide This is “real” >> multithreading." >> >> if he is right(QThread is "real" multithreading in Python),then why not >> add QtConcurrent to PySide? The documentation doesn't appear to show any >> of the functions available in the QtConcurrent namespace >> >> >> _______________________________________________ >> PySide mailing list >> [email protected] >> http://lists.qt-project.org/mailman/listinfo/pyside >> >> >
_______________________________________________ PySide mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/pyside
