Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread David Beazley
I have been following this discussion about fixing the GIL and just wanted to make a few comments about it. To the doubters who don't think this is a real problem worth fixing, I must respectfully disagree. Multicore systems and parallel programming issues are not going away any time in the

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Bill Janssen
Antoine Pitrou wrote: > Le Sun, 14 Mar 2010 12:37:48 PDT, > Bill Janssen a écrit : > > > > The old GIL is causing me a great deal of grief. My server is heavily > > threaded, and when I get a number of Python 2.6 threads doing layout > > analysis, my 4 and 8 core machines turn into slow thrash

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Gregory P. Smith
On Sun, Mar 14, 2010 at 4:31 AM, Nir Aides wrote: > There are two possible problems with Dave's benchmark: > 1) On my system setting TCP_NODELAY option on the accepted server socket > changes results dramatically. > 2) What category of socket servers is dave's spin() function intended to > simulat

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Greg Ewing
Jon Ribbens wrote: Sorry if I missed it, but why on earth is the bytecode directory __pycache__ and not .pycache? (Or indeed anything else that starts with a '.') Surely this is a classic ideal use case for a "hidden" directory? Please don't try to hide it. On MacOSX, it makes it impossible to

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Martin v. Löwis
> Testing with a modified server that reflects the above indicates the new > GIL behaves just fine in terms of throughput. > So a change to the GIL may not be required at all. Thanks - I think you put it better than I did. Regards, Martin ___ Python-Dev

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Antoine Pitrou
Le Sun, 14 Mar 2010 23:11:44 +0200, Nir Aides a écrit : > > I first disabled the call to spin() but client running time remained > around 30 seconds. > I then added TCP_NODELAY and running time dropped to a few dozen > milliseconds for the entire no-spin run. You don't want the benchmark to be d

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Antoine Pitrou
Le Sun, 14 Mar 2010 12:37:48 PDT, Bill Janssen a écrit : > > The old GIL is causing me a great deal of grief. My server is heavily > threaded, and when I get a number of Python 2.6 threads doing layout > analysis, my 4 and 8 core machines turn into slow thrashers. Have you checked whether the 2

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Nick Coghlan
R. David Murray wrote: > On Sun, 14 Mar 2010 12:59:50 -, Jon Ribbens > wrote: >> Sorry if I missed it, but why on earth is the bytecode directory >> __pycache__ and not .pycache? (Or indeed anything else that starts >> with a '.') Surely this is a classic ideal use case for a "hidden" >> dire

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Nir Aides
inline: On Sun, Mar 14, 2010 at 3:54 PM, Peter Portante wrote: > On 3/14/10 7:31 AM, "Nir Aides" wrote: > > There are two possible problems with Dave's benchmark: > > 1) On my system setting TCP_NODELAY option on the accepted server socket > changes results dramatically. > > Could you document

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Bill Janssen
Cameron Simpson wrote: > Currently all the above is somewhat "batchy"; the client end looks like > an archiving command line tool, but it's intended to have a FUSE mode to > present the archive as a filesystem; that I can imagine tripping over > this issue as a user uses the filesystem for "stuff

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Martin v. Löwis
> Speaking for myself, I have an app with a daemon mode which I expect > will sometimes behave as described; it answers requests and thus has I/O > bound threads waiting for requests and dispatching replies, and threads > doing data handling, which make constant use of the zlib library. This is th

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread R. David Murray
On Sun, 14 Mar 2010 12:59:50 -, Jon Ribbens wrote: > On Fri, Mar 12, 2010 at 05:56:57PM -0500, Barry Warsaw wrote: > > Nope, sorry I should have been clearer. > > > > *creation* is the key here. As per BDFL pronouncement, we'll support > > reading > > pyc-only modules just like we do today

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Paul Moore
On 13 March 2010 14:17, Nick Coghlan wrote: > The creation side could be made a little more explicit in the PEP. We > could also do something via the compileall module. > > (Pause while Nick goes and reads the source code for compileall for the > first time ever...) > > Hmm - methinks the PEP actu

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Antoine Pitrou
Le Sun, 14 Mar 2010 12:59:50 +, Jon Ribbens a écrit : > > Sorry if I missed it, but why on earth is the bytecode directory > __pycache__ and not .pycache? (Or indeed anything else that starts > with a '.') Surely this is a classic ideal use case for a "hidden" > directory? I suppose the same

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Steven D'Aprano
On Sun, 14 Mar 2010 11:59:50 pm Jon Ribbens wrote: > Sorry if I missed it, but why on earth is the bytecode directory > __pycache__ and not .pycache? (Or indeed anything else that starts > with a '.') Surely this is a classic ideal use case for a "hidden" > directory? I disagree with your assumpt

Re: [Python-Dev] __file__ and bytecode-only

2010-03-14 Thread Jon Ribbens
On Fri, Mar 12, 2010 at 05:56:57PM -0500, Barry Warsaw wrote: > Nope, sorry I should have been clearer. > > *creation* is the key here. As per BDFL pronouncement, we'll support reading > pyc-only modules just like we do today. This is in PEP 3147. We won't > support creating them though. > > B

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Peter Portante
On 3/14/10 7:31 AM, "Nir Aides" wrote: > There are two possible problems with Dave's benchmark: > > 1) On my system setting TCP_NODELAY option on the accepted server socket > changes results dramatically. Could you document what you saw and explain how you think TCP_NODELAY makes a difference, i

Re: [Python-Dev] C++

2010-03-14 Thread Andrew MacIntyre
Jeffrey Yasskin wrote: On Fri, Mar 12, 2010 at 7:54 PM, wrote: OS/2: http://www.andymac.org/ I can't find a modern c++ compiler for OS/2. Then again, your link only provides python 2.4. While the gcc used as part of the EMX toolchain is 2.8.1, there are ports of gcc 3.3.5 and 4.4.x us

Re: [Python-Dev] "Fixing" the new GIL

2010-03-14 Thread Nir Aides
There are two possible problems with Dave's benchmark: 1) On my system setting TCP_NODELAY option on the accepted server socket changes results dramatically. 2) What category of socket servers is dave's spin() function intended to simulate? In a server which involves CPU intensive work in respons