On 5/5/07, tomer filiba <[EMAIL PROTECTED]> wrote: > i have to admit i've been neglecting the list in the past few months, > and i don't know whether the issue i want to bring up has been > settled already.
It's been settled by default -- nobody submitted a PEP to kill the GIL in time for the April 30 deadline, and I won't accept one now. > as you all may have noticed, multicore processors are becoming > more and more common in all kinds of machines, from desktops > to servers, and will surely become more prevalent with time, > as all major CPU vendors plan to ship 8-core processors > by mid-2008. > > back in the day of uniprocessor machines, having the GIL really > made life simpler and the sacrifice was negligible. > > however, running a threaded python script over an 8-core > machine, where you can utilize at most 12.5% of the horsepower, > seems like too large a sacrifice to me. > > the only way to overcome this with cpython is to Kill The GIL (TM), > and since it's a very big architectural change, it ought to happen > soon. pushing it further than version 3.0 means all library authors > would have to adapt their code twice (once to make it compatible > with 3.0, and then again to make it thread safe). Here's something I wrote recently to someone (a respected researcher) who has a specific design in mind to kill the GIL (rather than an agenda without a plan). """ Briefly, the reason why it's so hard to get rid of the GIL is that this Python implementation uses reference counting as its primary GC approach (there's a cycle-traversing GC algorithm bolted on the side, but most objects are reclaimed by refcounting). In Python, everything is an object (even small integers and characters), and many objects are conceptually immutable, allowing free sharing of objects as values between threads. There is no marking of objects as "local to a thread" or "local to a frame" -- that would be a totally alien concept. All objects have a refcount field (a long at the front of the object structure) and this sees a lot of traffic. As C doesn't have an atomic increment nor an atomic decrement-and-test, the INCREF and DECREF macros sprinkled throughout the code (many thousands of them) must be protected by some lock. Around '99 Greg Stein and Mark Hammond tried to get rid of the GIL. They removed most of the global mutable data structures, added explicit locks to the remaining ones and to individual mutable objects, and actually got the whole thing working. Unfortunately even on the system with the fastest locking primitives (Windows at the time) they measured a 2x slow-down on a single CPU due to all the extra locking operations going on. Good luck fixing this! My personal view on it is that it's not worth it. If you want to run Python on a multiprocessor, you're much better off finding a way to break the application off into multiple processes each running a single CPU-bound thread and any number of I/O-bound threads; alternatively, if you cannot resist the urge for multiple CPU-bound threads, you can use one of the Python implementations built on inherently multi-threading frameworks, i.e. Jython or IronPython. But I'd be happy to be proven wrong, if only because this certainly is a recurring heckle whenever I give a talk about Python anywhere. """ -- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
