"Andre Meyer" <[EMAIL PROTECTED]> wrote: > Dear Python experts > > As a heavy user of multi-threading in Python and following the current > discussions about Python on multi-processor systems on the python-list I > wonder what the plans are for improving MP performance in Py3k. MP systems > become more and more common as most modern processors have multiple > processing units that could be used in parallel by distributing threads. > Unfortunately, the GIL in CPython prevents to use this mechanism. As far as > I understand IronPython, Jython and PyPy do not suffer from this. > > While I understand the difficulties in removing the GIL and the potential > negative effect on single-threaded applications I would very much encourage > discussion to seriously consider removing the GIL (maybe optionally) in > Py3k. If not, what alternatives would you suggest?
Search for 'Python free threading' without quotes in Google to find the discussions about this topic over the years. Personally, I think that the effort to remove the GIL in Py3k (or otherwise) is quite a bit of trouble that we don't want to have to go through; both from an internal redesign, and C-extension perspective. It would be substantially easier if there were a distributed RPC mechanism that auto distributed to the "least-working" process in a set of potential working processes on a single machine. Something with the simplicity of XML-RPC calling (but without servers and clients) and the distribution properties of Linda. Of course then we run into a situation where we need to "pickle" the callable arguments across a connection of some kind. There is a solution to this on a single machine; copying the internal representation of every object in the arguments of a function call to memory shared between all processes (mmap). With such a semantic, only mutable portions need to be copied out into non-mmap memory. With that RPC mechanism and file handle migration (available on BSDs natively, linux with minor work, and Windows via pywin32), most operations would *just work*, would be reasonably fast, and Python could keep its GIL - which would be substantially less work for everyone involved. The details are cumbersome (specifically the copying Python objects to/from memory), but they can be made less cumbersome if one only allows builtin objects to be transferred. - Josiah _______________________________________________ Python-3000 mailing list [email protected] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
