Re: [Python-Dev] Python parallel benchmark

2008-05-16 Thread Matthieu Brucher
2008/5/16 Hrvoje Nikšić <[EMAIL PROTECTED]>: > On Fri, 2008-05-16 at 08:04 -0400, Tom Pinckney wrote: > > Here's one example, albeit from a few years ago > > > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 > > Thanks for the pointer. I'm not sure I fully understand Konr

Re: [Python-Dev] Python parallel benchmark

2008-05-16 Thread Hrvoje Nikšić
On Fri, 2008-05-16 at 08:04 -0400, Tom Pinckney wrote: > Here's one example, albeit from a few years ago > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 Thanks for the pointer. I'm not sure I fully understand Konrad Hinsen's concerns, but maybe the problem is that Nump

Re: [Python-Dev] Python parallel benchmark

2008-05-16 Thread Tom Pinckney
Here's one example, albeit from a few years ago http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 But, I am a numpy novice and so no idea what it actually does in its current form. On May 16, 2008, at 4:17 AM, Hrvoje Nik?i? wrote: On Thu, 2008-05-15 at 21:02 -0400, Tom

Re: [Python-Dev] Python parallel benchmark

2008-05-16 Thread Hrvoje Nikšić
On Thu, 2008-05-15 at 21:02 -0400, Tom Pinckney wrote: > I found some other references where people were expressing concern > over numpy releasing the GIL due to the fact that other C extensions > could call numpy and unexpectedly have the GIL released on them (or > something like that). Cou

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
Interestingly, I think there's something magic going on with numpy.dot() on my mac. If I just run a program without threading--that is just a numpy matrix multiply such as: import numpy a = numpy.empty((4000,4000)) b = numpy.empty((4000,4000)) c = numpy.dot(a,b) then I see both cores fully

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Greg Ewing
Tom Pinckney wrote: If I look at top while running 2 or more threads, both cores are being used 100% and there is no idle time on the system. If you run it with just one thread, does it use up only one core's worth of CPU? If so, this suggests that the GIL is being released. If it wasn't, two

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
I switched to using numpy for the matrix multiply and while the overall time to do the matrix multiply is much faster, there is still no speed up from using more than one python thread. If I look at top while running 2 or more threads, both cores are being used 100% and there is no idle tim

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Jesse Noller
On May 15, 2008, at 6:54 PM, Eric Smith [EMAIL PROTECTED]> wrote: Jesse Noller wrote: Do you have the code posted someplace for this? I'd like to add it into the tests I am running It would also be interesting to see how pyprocessing performs. Eric. I'm working on exactly that - I have

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Eric Smith
Jesse Noller wrote: Do you have the code posted someplace for this? I'd like to add it into the tests I am running It would also be interesting to see how pyprocessing performs. Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.pyth

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Jesse Noller
Do you have the code posted someplace for this? I'd like to add it into the tests I am running On May 15, 2008, at 11:56 AM, Tom Pinckney <[EMAIL PROTECTED]> wrote: All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performan

Re: [Python-Dev] Python parallel benchmark

2008-05-15 Thread Nick Coghlan
Tom Pinckney wrote: All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performance to see if reality matched my expectations around what would scale up and what would not. I knew Python threads wouldn't be good for compute bound p

[Python-Dev] Python parallel benchmark

2008-05-15 Thread Tom Pinckney
All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performance to see if reality matched my expectations around what would scale up and what would not. I knew Python threads wouldn't be good for compute bound problems, but I wa