On Wednesday, April 30, 2014 3:52:49 PM UTC-4, Joachim Durchholz wrote: > > Am 30.04.2014 19:13, schrieb Vinzent Steinberg: > > Python does not support running threads in parallel because of the GIL. > > Not quite so - there's actually threading modules in Python. >
Python's threading can only use one core. [1] It is useful if you want to avoid IO blocking. It cannot execute things in parallel. The GIL does not prevent multithreading per se, it just prevents > parallel bytecode execution. For programs like SymPy which are > bytecode-heavy, this means that you don't gain performance (but there > can be improved program structure for some kinds of tasks). > I don't understand how that makes any difference. Don't we want to execute bytecode in parallel? I think the only way to release the GIL is by calling external code, which is what I meant by "escaping to other languages". > Now the GIL isn't present in all Python implementations. > ipython and parallelpython were mentioned elsewhere, and I believe that > Stackless Python doesn't have it either. > ipython and parallelpython are Python implementations? I think they just use multiple processes, like CPython's multiprocessing. Jython, IronPython don't have a GIL. Stackless Python does. However, Sympy is targeting CPython. > Also, Python does come with multithreading facilities in the standard > library, and I doubt that they would be there if the GIL were really the > end-all of any hope of having multithreading in Python. > Even with the GIL, threading can be useful. (I.e. if you are waiting for IO or writing a GUI.) And, yes, I believe the GIL is the end of all hope to execute threads in parallel within the same Python process. > > It > > can only run processes in parallel, which adds a large overhead, because > > memory cannot be shared. > > Read-only data should be sharable. > Only among threads, not among processes. Your OS should not allow you to read another's process memory. (In theory, os.fork() is often implemented efficiently using copy-on-write, but I imagine it is hard to share Python data types without triggering the copy. See [2] for a few options on how to do it for binary data.) > Though I guess that the extremely dynamic nature of Python makes it hard > for the interpreter to identify read-only data, so everything ever > shared across threads would undergo frequent locking and unlocking > operations. > I'm only guessing here though. > > > This means that all data shared between processes > > has to be serialized (usually using pickle). > > For really slow algorithms that can take hours to complete (Risch etc.), > even that may pay off. > SymPy's data structures aren't very large, the pickling overhead would > be negligible for slow algorithms. > I think it depends on how parallelizable your algorithm is. Without knowing where the Risch algorithm spends most of the time, I doubt it is easily parallelizable. Are there actually examples where Risch runs for hours and produces useful results? The larger problem would be to find out how reliable operation is > possible. I have little knowledge of how the various Python > implementations differ wrt. parallel execution; we might find that we > can't easily make SymPy work reliably on all relevant platforms. We > might also find that it's too much effort to get a parallel and a > sequential version of an algorithm to work equivalently, and keep them > that way while SymPy undergoes improvements and changes. > Vinzent [1] https://docs.python.org/2/library/threading.html [2] https://stackoverflow.com/questions/17785275/share-large-read-only-numpy-array-between-multiprocessing-processes -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/3fbcadc7-18c3-4cc2-a6cc-4785ace133d5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
