Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Paul Moore
2008/12/17 Greg Ewing greg.ew...@canterbury.ac.nz: Nick Coghlan wrote: Actually, I believe 3.0 already took a big step towards allowing this by changing the way modules are initialised. It's a step, but I wouldn't call it a big one. There are many other problems to be solved before fully

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Greg Ewing
Paul Moore wrote: Do you know if these remaining problems are listed anywhere? There was a big discussion about this in comp.lang.python not long ago. Basically all the built-in types and constants are shared between interpreters, which means you still need a GIL to stop different interpreters

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Nick Coghlan
Greg Ewing wrote: Paul Moore wrote: Do you know if these remaining problems are listed anywhere? There was a big discussion about this in comp.lang.python not long ago. Basically all the built-in types and constants are shared between interpreters, which means you still need a GIL to stop

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Paul Moore
2008/12/18 Greg Ewing greg.ew...@canterbury.ac.nz: Paul Moore wrote: Do you know if these remaining problems are listed anywhere? There was a big discussion about this in comp.lang.python not long ago. Basically all the built-in types and constants are shared between interpreters, which

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Christian Heimes
Paul Moore schrieb: OK, but how close is it to providing isolation for threads running under the control of the GIL? I'm thinking of something along the lines of an in-process version of fork(), which spawns a new interpreter and runs the 2 interpreters as threads, still using the GIL to

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-18 Thread Martin v. Löwis
OK, but how close is it to providing isolation for threads running under the control of the GIL? They won't be indedepent. If an extension module has a global variable, that will be shared across interpreters. If that variable supports modifiable state, such modifications will leak across

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-17 Thread Greg Ewing
Nick Coghlan wrote: Actually, I believe 3.0 already took a big step towards allowing this by changing the way modules are initialised. It's a step, but I wouldn't call it a big one. There are many other problems to be solved before fully independent interpreters are possible. -- Greg

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-16 Thread Ivan Krstić
On Dec 13, 2008, at 5:47 PM, Martin v. Löwis wrote: They were originally invented in 1965, on Multics (1970) they were used to perform compilation in the background. When Unix came along, it *added* address space separation, introducing what is now known as processes. Yes, and a lot of

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Michael Foord
Lennart Regebro wrote: On Fri, Dec 12, 2008 at 02:13, Sturla Molden stu...@molden.no wrote: I genuinely think the use of threads should be discouraged. It leads to code that are full of bugs and difficult to maintain - race conditions, deadlocks, and livelocks are common pitfalls. The

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Guido van Rossum
Yes, this is what threads were designed for. As an abstraction to have multiple threads of control on a *single* processor (in a single process). The whole multi-core business came decades later. (Classic multi-processors have something called threads too, but they, too, came later than the

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Steve Holden
If I remember correctly (when threading was invented in the mid-1980s) threads were originally described as lightweight processes. The perceived advantage at the time was the ability to have multiple threads of control with shared memory: this was much faster than the available inter-process

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Christian Heimes
Steve Holden schrieb: If I remember correctly (when threading was invented in the mid-1980s) threads were originally described as lightweight processes. The perceived advantage at the time was the ability to have multiple threads of control with shared memory: this was much faster than the

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Guido van Rossum
On Sat, Dec 13, 2008 at 9:13 AM, Christian Heimes li...@cheimes.de wrote: Steve Holden schrieb: If I remember correctly (when threading was invented in the mid-1980s) threads were originally described as lightweight processes. The perceived advantage at the time was the ability to have

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-13 Thread Martin v. Löwis
If I remember correctly (when threading was invented in the mid-1980s) threads were originally described as lightweight processes. According to http://www.serpentine.com/blog/threads-faq/the-history-of-threads/ that's when threads where *reinvented*. They were originally invented in 1965, on

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Stefan Behnel
Hi, replying to the topic only: because many C libraries support threading and Python extension modules can integrate them in a way that allows concurrency in a safe way (although 'safe' is definitely something that is paid for in developer days). Stefan

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Paul Moore
2008/12/12 Sturla Molden stu...@molden.no: Last month there was a discussion on Python-Dev regarding removal of reference counting to remove the GIL. I hope you forgive me for continuing the debate. [...] Python could be better off doing what tcl does. Allow each process to embed multiple

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Lennart Regebro
On Fri, Dec 12, 2008 at 02:13, Sturla Molden stu...@molden.no wrote: I genuinely think the use of threads should be discouraged. It leads to code that are full of bugs and difficult to maintain - race conditions, deadlocks, and livelocks are common pitfalls. The use of threads for load

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Sturla Molden
On 12/12/2008 11:52 AM, Lennart Regebro wrote: The use of threads for load balancing should be discouraged, yes. That is not what they are designed for. Threads are designed to allow blocking processes to go on in the background without blocking the main process. It seems that most

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Nick Coghlan
Sturla Molden wrote: Last month there was a discussion on Python-Dev regarding removal of reference counting to remove the GIL. I hope you forgive me for continuing the debate. Anything to do with removing the GIL/threads/whatever other core language feature someone doesn't like really belongs

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Nick Coghlan
Paul Moore wrote: 2. I'd like to see isolation based on multiple interpreters, but the problem lies with extensions (and at a lower level with the Python C API) which wasn't designed with isolation in mind. Changing that may be nice, but it's probably too late (or if not, it's likely to be a

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Christian Heimes
Nick Coghlan schrieb: Actually, I believe 3.0 already took a big step towards allowing this by changing the way modules are initialised. You are believing correctly. Martin has designed and implemented a nicely working API to store extension module data per interpreter state. For now

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Thomas Heller
Christian Heimes schrieb: Nick Coghlan schrieb: Actually, I believe 3.0 already took a big step towards allowing this by changing the way modules are initialised. You are believing correctly. Martin has designed and implemented a nicely working API to store extension module data per

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Lennart Regebro
On Fri, Dec 12, 2008 at 12:23, Sturla Molden stu...@molden.no wrote: It seems that most programmers with Java or Windows experience don't understand this; hence the ever lasting GIL debate. Yes. Maybe writing this with big letters in the thread module docs would help? I am not suggesting

Re: [Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-12 Thread Nick Coghlan
Thomas Heller wrote: Christian Heimes schrieb: Nick Coghlan schrieb: Actually, I believe 3.0 already took a big step towards allowing this by changing the way modules are initialised. You are believing correctly. Martin has designed and implemented a nicely working API to store extension

[Python-Dev] The endless GIL debate: why not remove thread support instead?

2008-12-11 Thread Sturla Molden
Last month there was a discussion on Python-Dev regarding removal of reference counting to remove the GIL. I hope you forgive me for continuing the debate. I think reference counting is a good feature. It prevents huge piles of garbage from building up. It makes the interpreter run more smoothly.