Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-23 Thread Michael Hudson
Gregory P. Smith [EMAIL PROTECTED] writes: On Mon, Sep 19, 2005 at 09:12:05PM +0100, Michael Hudson wrote: Martin Blais [EMAIL PROTECTED] writes: http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software Herb Sutter

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-23 Thread Paul Moore
On 9/21/05, Josiah Carlson [EMAIL PROTECTED] wrote: Antoine Pitrou [EMAIL PROTECTED] wrote: The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming.

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Antoine Pitrou
Hi, On *nix, use a unix domain socket (location in the filesystem which acts as a listening socket). On Windows, you can use cTypes, pywin32, etc., to create a global mutex and/or COM interface. Ok, but for a very simple and crude need like mine (the application code using this IPC means is

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Oleg Broytmann
On Thu, Sep 22, 2005 at 10:27:10AM +0400, Sokolov Yura wrote: MULTIPROCESSING RULES!!! Everything in programming is about divide and conquer. Separate and control. Modules. Classes. Namespaces. And now that multithreading with shared memory. That's an evil idea, it causes a lot of

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Phillip J. Eby
At 10:27 AM 9/22/2005 +0400, Sokolov Yura wrote: It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. It seems you've never heard of fork(), which works just fine to scale Python processes on multiprocessor boxes. I've actually done this,

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Fredrik Lundh
Phillip J. Eby wrote: It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. It seems you've never heard of fork(), which works just fine to scale Python processes on multiprocessor boxes. there's a version of fork hidden somewhere in

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Phillip J. Eby
At 08:42 PM 9/22/2005 +0200, Fredrik Lundh wrote: Phillip J. Eby wrote: At 10:27 AM 9/22/2005 +0400, Sokolov Yura wrote: It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. It seems you've never heard of fork(), which works just fine to

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Bill Janssen
Sokolov Jura writes: It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. CPython will not be wide popular without real multithreading. He's right. Bill ___ Python-Dev mailing list

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-22 Thread Terry Reedy
Bill Janssen [EMAIL PROTECTED] wrote in message news:05Sep22.141518pdt.58617@synergy1.parc.xerox.com... Sokolov Jura writes: It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. CPython will not be wide popular without real

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Guido van Rossum
On 9/21/05, Donovan Baarda [EMAIL PROTECTED] wrote: The reality is threads were invented as a low overhead way of easily implementing concurrent applications... ON A SINGLE PROCESSOR. Taking into account threading's limitations and objectives, Python's GIL is the best way to support threads.

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Simon Percivall
On 21 sep 2005, at 12.33, Donovan Baarda wrote: In the short term there will be various hacks to try and make the existing plethora of threading applications run better on multiple processors, but ultimately the overheads of shared memory will force serious multi-processing to use IPC

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Antoine Pitrou
The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. 100% agreed. I needed a portable way to know if a program was already running (and then to send it a

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Jonathan LaCour
The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. +100 Huge amounts of effort would have to be invested to remove the GIL for the benefit of threads. Not

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Josiah Carlson
Antoine Pitrou [EMAIL PROTECTED] wrote: The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. 100% agreed. I needed a portable way to know if a

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Guido van Rossum
(Interestign to see the PyObjC folks disagree. :-) On 9/21/05, Ronald Oussoren [EMAIL PROTECTED] wrote: On 21-sep-2005, at 0:10, Bob Ippolito wrote: My use case for this isn't so much about threads, but plug-ins. Writing multiple Python-based plug-ins for an application is always a

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Thomas Heller
Bob Ippolito [EMAIL PROTECTED] writes: Personally my biggest issue with the way the CPython VM works is that you can't reliably do multiple interpreters in a single process. If that worked well, you could start an independent interpreter per thread and with a per-interpreter GIL you'd

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Phillip J. Eby
At 12:04 PM 9/21/2005 -0700, Guido van Rossum wrote: Actually Python itself has a hard time keeping multiple interpreters truly separate. Also of course there are some shared resources maintained by the operating system: current directory, open file descriptors, signal settings, child processes,

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Bill Janssen
The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. And the model provided by the thread abstraction is a good API for that support... Bill

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-21 Thread Josiah Carlson
Bill Janssen [EMAIL PROTECTED] wrote: The best way to make people stop complaining about the GIL and start using process-based multiprogramming is to provide solid, standardized support for process-based multiprogramming. And the model provided by the thread abstraction is a good

Re: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.)

2005-09-20 Thread John J Lee
On Sun, 18 Sep 2005, Guido van Rossum wrote: On 9/17/05, John J Lee [EMAIL PROTECTED] wrote: [...snip...] [guido] If my hunch is right, I expect that instead of writing massively parallel applications, we will continue to write single-threaded applications that are tied together at the

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread John J Lee
On Mon, 19 Sep 2005, Florian Weimer wrote: The real problem is that you can ditch most extension modules. 8-( [...] *Is* that a showstopper for Python 3.0, though? John ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.)

2005-09-20 Thread John J Lee
On Tue, 20 Sep 2005, John J Lee wrote: [...] I don't actively want a GIL-free Python. I was just making some arguments [...] Actually, FWIW, I don't know if I even *passively* want a GIL-free Python, if it encourages threaded code (though I'd like to have that option for my occasional personal

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread Brett Cannon
On 9/20/05, John J Lee [EMAIL PROTECTED] wrote: On Mon, 19 Sep 2005, Florian Weimer wrote: The real problem is that you can ditch most extension modules. 8-( [...] *Is* that a showstopper for Python 3.0, though? Who knows. I bet Guido doesn't even know how much breakage he is going to

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread John J Lee
On Mon, 19 Sep 2005, Tim Lesher wrote: On 9/19/05, Michael Hudson [EMAIL PROTECTED] wrote: I was disappointed that that article (hey, it was the only issue of ddj I've ever actually bought! :) didn't consider any concurrency models other than shared memory threading. The problem is

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread John J Lee
On Tue, 20 Sep 2005, Brett Cannon wrote: On 9/20/05, John J Lee [EMAIL PROTECTED] wrote: On Mon, 19 Sep 2005, Florian Weimer wrote: The real problem is that you can ditch most extension modules. 8-( [...] *Is* that a showstopper for Python 3.0, though? Who knows. I bet Guido

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread Guido van Rossum
On 9/20/05, John J Lee [EMAIL PROTECTED] wrote: threading is not the only, nor the best, concurrency model. But maybe these chips designed with threading in mind blow that argument out of the water. I don't know enough to know whether that's true or not... I don't know that any chips are

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread Bob Ippolito
On Sep 20, 2005, at 5:43 PM, Guido van Rossum wrote: On 9/20/05, John J Lee [EMAIL PROTECTED] wrote: threading is not the only, nor the best, concurrency model. But maybe these chips designed with threading in mind blow that argument out of the water. I don't know enough to know whether

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-20 Thread Phillip J. Eby
At 06:10 PM 9/20/2005 -0400, Bob Ippolito wrote: My use case for this isn't so much about threads, but plug-ins. Writing multiple Python-based plug-ins for an application is always a mess, because they share too much (sys.modules, sys.path, etc.). PyObjC would benefit greatly from this feature,

Re: [Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.)

2005-09-19 Thread Martin Blais
On 9/18/05, Guido van Rossum [EMAIL PROTECTED] wrote: On 9/17/05, John J Lee [EMAIL PROTECTED] wrote: c. Since time is needed to iron out bugs (and perhaps also to reimplememt some pieces of code from scratch), very early in the life of Python 3 seems like the least-worst time to

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Michael Hudson
Martin Blais [EMAIL PROTECTED] writes: On 9/18/05, Guido van Rossum [EMAIL PROTECTED] wrote: On 9/17/05, John J Lee [EMAIL PROTECTED] wrote: c. Since time is needed to iron out bugs (and perhaps also to reimplememt some pieces of code from scratch), very early in the life of Python 3

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Tim Lesher
On 9/19/05, Michael Hudson [EMAIL PROTECTED] wrote: I was disappointed that that article (hey, it was the only issue of ddj I've ever actually bought! :) didn't consider any concurrency models other than shared memory threading. The problem is that, for all its limitations, shared-memory

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Florian Weimer
* Martin Blais: http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software Herb Sutter March 2005 This piece is fundamentally wrong. We all have been writing concurrent server-side software for eons. I don't know what Herb

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Florian Weimer
* Guido van Rossum: That assumes a very specific model for how all that MP power is going to be used. Indeed. I personally don't think the threaded programming model as found in Java works all that well; without locks you end up with concurrent modification errors, with locks you get

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Michael Hudson
Florian Weimer [EMAIL PROTECTED] writes: By the way, has anybody ever tried to create a CPython variant which uses a (mostly) copying garbage collector (or something else except reference counting or Boehm GC)? Not to my knowledge. I've always thought that it would be pretty hard. I'd be

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Florian Weimer
* Michael Hudson: Not to my knowledge. I've always thought that it would be pretty hard. I'd be interested in being proved wrong. The real problem is that you can ditch most extension modules. 8-( It sounds more like a fun project for the Python core, though. Copying GC might help to get

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Scott David Daniels
Michael Hudson wrote: How does a copying gc differ much from a non-copying non-refcounted gc here? One important issue for C coded modules is that addresses may change when a GC is invoked, so no remembering addresses in your module; you must recalculate before each use. -- Scott David

Re: [Python-Dev] GIL, Python 3, and MP vs. UP

2005-09-19 Thread Barry Warsaw
On Mon, 2005-09-19 at 17:52, Scott David Daniels wrote: Michael Hudson wrote: How does a copying gc differ much from a non-copying non-refcounted gc here? One important issue for C coded modules is that addresses may change when a GC is invoked, so no remembering addresses in your