thanks for your experienced suggestion,but what do you mean by Cython has
nothing to do with parallelism ?have you visited this page ?
http://docs.cython.org/src/userguide/parallelism.html
BTW ,I have no experience with Cython :)
------------------ Original ------------------
From: "Sean Fisk";<[email protected]>;
Date: Mon, Jan 13, 2014 02:36 AM
To: "iMath"<[email protected]>;
Cc: "pyside"<[email protected]>;
Subject: Re: [PySide] QtConcurrent support?
On Sun, Jan 12, 2014 at 4:32 AM, iMath <[email protected]> wrote:
thanks for your great explanation ,it helped me a lot !!!
the GIL is not present under Ironpython nor Jython and so, running
multithreaded Python code under them WILL give the expected performance
benefits .
That is correct. However, if you still want to use CPython, can also avoid the
GIL with multiple processes using multiprocessing or
concurrent.futures.ProcessPool. I have also had a good experience with Cython,
which of course has nothing to do with parallelism.
A short aside: I studied Python for high-performance computing in university
and also at my internship with NCAR. My opinion: don??t use it, or move your
performance-sensitive code to C/C++ extensions or Cython. There are a myriad of
solutions out there that boil down to one of four things: fork(), pthreads,
MPI, or CUDA. Tacking on one of [multiprocessing, multithreading,
message-passing, or stream processing] to Python and expecting it to be fast is
like fitting a square peg in a round hole. It??s a great academic exercise, but
not very practical.
That??s just my opinion, so of course take it with a grain of salt. I love
Python, hence being on this list. But I don??t love it for it??s performance :)
http://www.neotitans.com/resources/python/python-threads-multithreading-tutorial.html
------------------ Original ------------------
From: "Sean Fisk";<[email protected]>;
Date: Sun, Jan 12, 2014 02:30 PM
To: "iMath"<[email protected]>;
Cc: "pyside"<[email protected]>;
Subject: Re: [PySide] QtConcurrent support?
Hello iMath,
This is a complicated topic; one that I have been studying for the past six
months or so. I would specifically avoid the term ??real multithreading.??
There are enough factors that come into play to make that term virtually
unqualified.
The GIL and Parallelism
What it boils down to is this:
If Python code is running, the GIL is in effect.
If Python code is not running (e.g., C, C++), the GIL may or may not be in
effect.
Instead of real multithreading, let us instead define the term true in-process
parallelism as follows:
Given that a process has multiple instructions, true in-process parallelism
occurs when two or more instructions are executed in the same instant.
Obviously, a machine would need to possess multiple processors for this to
happen. But true in-process parallelism is specifically what the GIL prevents.
Hence, no matter whether you use threading or QThread or anything else built on
these, the GIL will prevent true in-process parallelism for pure Python code.
However, just because two instructions can??t happen in the same instant in
Python code doesn??t make threading useless. If you want two operations to
appear to be executing simultaneously, threading in Python will suffice.
However, you won??t get any speed boost because your code can??t truly execute
simultaneously, thanks to the GIL.
Extension Modules
Extension modules (e.g., written in C, C++) complicate the issue because they
have the possibility of releasing the GIL. PySide, as you know, is an extension
module. I can??t say much here, though, because AFAIK the only way to determine
whether an extension module is releasing the GIL is to look in its source code.
That??s somewhat out of my realm.
Just know that each time you call a PySide function which eventually leads to
C++ code (or, I guess, binary which was C++ code) being executed there is a
possibility of releasing the GIL and achieving true in-process parallelism.
QtConcurrent
As QtConcurrent is based on QThread, using it for a speedup [if it was in
PySide] would be basically moot. That??s because all of QtConcurrent??s
hypothetical functions take a callable as an argument, which, if written in
Python, would invoke the GIL.
In Python 2??s case, the much better options are to use the standard
multiprocessing module and concurrent.futures (which has been added to the
standard library in Python 3.2). Each of them provide a thread and process pool
approach. As implied earlier, only the process-based approach will allow for
the possibility of speedup (when in pure Python). These integrate much nicer
with Python and are probably better than what Qt could provide.
Bottom Line
What you choose depends on your use case. By the way, what are you trying to
achieve?
I don??t claim to be a threading or Python or GIL expert, but this is where my
research on the topic has pointed me. Please, someone correct me if you take
issue.
Sincerely,
--
Sean Fisk
On Sat, Jan 11, 2014 at 11:09 PM, iMath <[email protected]> wrote:
In these 2 posts
http://nathanhorne.com/?p=353
http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html
says
"when the threaded code is calling long running c/c++ functions, such as a
database select, it will release the GIL until the call completes. This is all
done automatically within python and PySide This is ??real?? multithreading."
if he is right(QThread is "real" multithreading in Python),then why not add
QtConcurrent to PySide? The documentation doesn't appear to show any of the
functions available in the QtConcurrent namespace
_______________________________________________
PySide mailing list
[email protected]
http://lists.qt-project.org/mailman/listinfo/pyside
_______________________________________________
PySide mailing list
[email protected]
http://lists.qt-project.org/mailman/listinfo/pyside
