Re: when normal parallel computations in CPython will be implemented at last?
John Nagle na...@animats.com wrote: It would be un-Pythonic to have real concurrency in Python. You wouldn't be able to patch code running in one thread from another thread. Some of the dynamic features of Python would break. If you want fine-grained concurrency, you need controlled isolation between concurrent tasks, so they interact only at well-defined points. That's un-Pythonic. I disagree. The situation in Python is no different than the situation in other programming languages. If you have shared state, you protect it with some kind of lock. After all, you don't patch code on a byte-by-byte basis -- you just change function bindings. That can be done atomically. -- Tim Roberts, t...@probo.com Providenza Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On 7/1/2012 10:51 AM, dmitrey wrote: hi all, are there any information about upcoming availability of parallel computations in CPython without modules like multiprocessing? I mean something like parallel for loops, or, at least, something without forking with copying huge amounts of RAM each time and possibility to involve unpiclable data (vfork would be ok, but AFAIK it doesn't work with CPython due to GIL). AFAIK in PyPy some progress have been done ( http://morepypy.blogspot.com/2012/06/stm-with-threads.html ) Thank you in advance, D. It would be un-Pythonic to have real concurrency in Python. You wouldn't be able to patch code running in one thread from another thread. Some of the dynamic features of Python would break. If you want fine-grained concurrency, you need controlled isolation between concurrent tasks, so they interact only at well-defined points. That's un-Pythonic. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
Dan Stromberg, 01.07.2012 21:28: On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans wrote: On 07/01/2012 08:44 PM, Dan Stromberg wrote: IronPython, sadly, lacks a python standard library. Beg pardon? https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib Perhaps things have changed. Yes, they have. The original restriction came from Microsoft enforcing a clean-room implementation for their code, which obviously excluded the standard library - which is well tested and licence cleared and all that, but nevertheless non-MS. When they dropped the IronPython project, it became free to integrate with other software. Clearly a cultural thing. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
If something happens with this for CPython, it'll likely come from Pypy developers first. They seem to be interested in doing things in a way that is (or can be made) compatible with CPython. If you want to help them along, they're taking donations to fund the work, or you could donate your own time. Bear in mind that CPython threads fine for I/O-heavy workloads. It's CPU-bound workloads that CPython doesn't thread super well. Last I heard, both Jython and IronPython thread well today, and there's little reason not to try them if you have a (rather rare) application that truly needs Java-like threading. Granted, they don't run C extension modules, but there's quite a bit of portability synergy to be had from coming up with Pure Python alternatives; it's quite feasible to run the same code on CPython 2.x, CPython 3.x, Pypy and Jython. IronPython, sadly, lacks a python standard library. If CPython has trouble with CoW, it's probably because of reference counting, not the GIL. Last I heard, CPython used both reference counting and garbage collection. Also, vfork is a bit of a hack... just plain fork is often a better way to go. On Sun, Jul 1, 2012 at 10:51 AM, dmitrey dmitre...@gmail.com wrote: hi all, are there any information about upcoming availability of parallel computations in CPython without modules like multiprocessing? I mean something like parallel for loops, or, at least, something without forking with copying huge amounts of RAM each time and possibility to involve unpiclable data (vfork would be ok, but AFAIK it doesn't work with CPython due to GIL). AFAIK in PyPy some progress have been done ( http://morepypy.blogspot.com/2012/06/stm-with-threads.html ) -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On 07/01/2012 07:51 PM, dmitrey wrote: hi all, are there any information about upcoming availability of parallel computations in CPython without modules like multiprocessing? I mean something like parallel for loops, or, at least, something without forking with copying huge amounts of RAM each time and possibility to involve unpiclable data (vfork would be ok, but AFAIK it doesn't work with CPython due to GIL). AFAIK in PyPy some progress have been done ( http://morepypy.blogspot.com/2012/06/stm-with-threads.html ) As far as I can tell, there are no concrete plans to integrate concurrency better, or get rid of the GIL, at the moment. To quote http://wiki.python.org/moin/GlobalInterpreterLock Getting rid of the GIL is an occasional topic on the python-dev mailing list. No one has managed it yet. There are currently no open or accepted PEPs on the subject of concurrency. http://www.python.org/dev/peps/ There is, of course, Stackless Python. http://stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On 07/01/2012 08:44 PM, Dan Stromberg wrote: IronPython, sadly, lacks a python standard library. Beg pardon? https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On 7/1/2012 1:53 PM, Thomas Jollans wrote: As far as I can tell, there are no concrete plans to integrate concurrency better, or get rid of the GIL, at the moment. To quote http://wiki.python.org/moin/GlobalInterpreterLock Getting rid of the GIL is an occasional topic on the python-dev mailing list. No one has managed it yet. There's also this, recently written by one of CPython's core devs: http://python-notes.boredomandlaziness.org/en/latest/python3/questions_and_answers.html#but-but-surely-fixing-the-gil-is-more-important-than-fixing-unicode -- CPython 3.3.0a4 | Windows NT 6.1.7601.17803 -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans t...@jollybox.de wrote: On 07/01/2012 08:44 PM, Dan Stromberg wrote: IronPython, sadly, lacks a python standard library. Beg pardon? https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib Perhaps things have changed. When I last checked the situation, IronPython came with no standard library, but you could bolt one on that hadn't been tested well - IIRC, just a simple import os gave a traceback. FePy was IronPython with a standard library and some degree of testing, but their emphasis was windows-only. I'd be open to using FePy instead, and I might even call it IronPython, but I'm not in the habit of happily using software that is Windows only. -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
On 07/01/2012 09:28 PM, Dan Stromberg wrote: On Sun, Jul 1, 2012 at 11:58 AM, Thomas Jollans t...@jollybox.de mailto:t...@jollybox.de wrote: On 07/01/2012 08:44 PM, Dan Stromberg wrote: IronPython, sadly, lacks a python standard library. Beg pardon? https://github.com/IronLanguages/main/tree/master/External.LCA_RESTRICTED/Languages/IronPython/27/Lib Perhaps things have changed. When I last checked the situation, IronPython came with no standard library, but you could bolt one on that hadn't been tested well - IIRC, just a simple import os gave a traceback. FePy was IronPython with a standard library and some degree of testing, but their emphasis was windows-only. I'd be open to using FePy instead, and I might even call it IronPython, but I'm not in the habit of happily using software that is Windows only. That must have been quite a while ago? I haven't tested it recently, but I'm fairly sure that IronPython includes a Python standard library which works reasonably well, and it's not Windows-only. (it works with Mono) -- http://mail.python.org/mailman/listinfo/python-list
Re: when normal parallel computations in CPython will be implemented at last?
Thomas Jollans t...@jollybox.de wrote: There is, of course, Stackless Python. http://stackless.com/ Stackless Python doesn't really address the original poster's problem as the GIL still effectively limits Python code running in one thread at a time. Ross Ridge -- l/ // Ross Ridge -- The Great HTMU [oo][oo] rri...@csclub.uwaterloo.ca -()-/()/ http://www.csclub.uwaterloo.ca/~rridge/ db // -- http://mail.python.org/mailman/listinfo/python-list
