Re: System, configuration and Python performance

2021-11-01 Thread Shaozhong SHI
On Tue, 2 Nov 2021 at 00:20, Shaozhong SHI wrote: > > > On Tue, 2 Nov 2021 at 00:09, MRAB wrote: > >> On 2021-11-01 23:02, Shaozhong SHI wrote: >> > How to configure to improve Python performance in a system like the >> > following: >> > >>

Re: System, configuration and Python performance

2021-11-01 Thread Chris Angelico
On Tue, Nov 2, 2021 at 11:21 AM Shaozhong SHI wrote: > > On Tue, 2 Nov 2021 at 00:09, MRAB wrote: > > > On 2021-11-01 23:02, Shaozhong SHI wrote: > > > How to configure to improve Python performance in a system like the > > > following: > >

Re: System, configuration and Python performance

2021-11-01 Thread Shaozhong SHI
On Tue, 2 Nov 2021 at 00:09, MRAB wrote: > On 2021-11-01 23:02, Shaozhong SHI wrote: > > How to configure to improve Python performance in a system like the > > following: > > > > Windows 10 > > > > System > > > > Processor Intel(R) Core(TM)

Re: System, configuration and Python performance

2021-11-01 Thread MRAB
On 2021-11-01 23:02, Shaozhong SHI wrote: How to configure to improve Python performance in a system like the following: Windows 10 System Processor Intel(R) Core(TM) i7-9700 CPU @3.60GHz, 3.60 GHz Installed memory (RAM) 32.0 GB (31.8 GB usable) System type: 64-bit Operating System, x64-based

System, configuration and Python performance

2021-11-01 Thread Shaozhong SHI
How to configure to improve Python performance in a system like the following: Windows 10 System Processor Intel(R) Core(TM) i7-9700 CPU @3.60GHz, 3.60 GHz Installed memory (RAM) 32.0 GB (31.8 GB usable) System type: 64-bit Operating System, x64-based processor I found that the Python script

[issue39168] Generic type subscription is a huge toll on Python performance

2020-11-15 Thread Guido van Rossum
Guido van Rossum added the comment: No worries. I tend to run each time it command at least three times before I trust the numbers. Professional bench markers also configure a machine without background tasks (email etc.). -- ___ Python tracker

[issue39168] Generic type subscription is a huge toll on Python performance

2020-11-15 Thread David Lord
David Lord added the comment: I'm using Arch Linux. After your reply I tried again and now I'm seeing the same result as you, negligible difference from inheriting `Generic` on Python 3.9. I can't explain it, I ran the timings repeatedly before I posted here, but I guess it was a weird

[issue39168] Generic type subscription is a huge toll on Python performance

2020-11-15 Thread Guido van Rossum
Guido van Rossum added the comment: @davidm I don't see such a dramatic difference -- the generic version is a tad slower, but the difference is less than the variation between runs. What platform are you using? (I'm doing this on Windows.) --

[issue39168] Generic type subscription is a huge toll on Python performance

2020-11-15 Thread David Lord
David Lord added the comment: Is this performance issue supposed to be fixed in 3.9? I'm still observing severe slowdown by inheriting from `Generic[T]`. I'm currently adding typing to Werkzeug, where we define many custom data structures such as `MultiDict`. It would be ideal for these

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-05 Thread Guido van Rossum
Change by Guido van Rossum : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-05 Thread miss-islington
miss-islington added the comment: New changeset 5a1384935ee8996a5bd240dd29f9b5e356cfc467 by Miss Islington (bot) in branch '3.9': bpo-39168: Remove the __new__ method of typing.Generic (GH-21327) https://github.com/python/cpython/commit/5a1384935ee8996a5bd240dd29f9b5e356cfc467 --

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-05 Thread miss-islington
Change by miss-islington : -- pull_requests: +20483 pull_request: https://github.com/python/cpython/pull/21335 ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-04 Thread miss-islington
miss-islington added the comment: New changeset 7fed75597fac11f9a6c769e2b6c6548fe0e4049d by Zackery Spytz in branch 'master': bpo-39168: Remove the __new__ method of typing.Generic (GH-21327) https://github.com/python/cpython/commit/7fed75597fac11f9a6c769e2b6c6548fe0e4049d -- nosy:

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-04 Thread Guido van Rossum
Guido van Rossum added the comment: Should this be backported? How far back? -- ___ Python tracker ___ ___ Python-bugs-list

[issue39168] Generic type subscription is a huge toll on Python performance

2020-07-04 Thread Zackery Spytz
Change by Zackery Spytz : -- keywords: +patch nosy: +ZackerySpytz nosy_count: 6.0 -> 7.0 pull_requests: +20477 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21327 ___ Python tracker

[issue39168] Generic type subscription is a huge toll on Python performance

2020-01-13 Thread Wouter De Borger
Change by Wouter De Borger : -- nosy: +Wouter De Borger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39168] Generic type subscription is a huge toll on Python performance

2020-01-11 Thread calebj
Change by calebj : -- nosy: +calebj ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39168] Generic type subscription is a huge toll on Python performance

2020-01-02 Thread Naveen Durai
Change by Naveen Durai : -- nosy: +Naveen Durai ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Ruslan Dautkhanov
Ruslan Dautkhanov added the comment: Didn't see your last response before submitting an update. That's great you have a plan how to resolve this! Thanks again -- ___ Python tracker

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Ruslan Dautkhanov
Ruslan Dautkhanov added the comment: Perhaps the check should only be done in some sort of Python development mode and off by default? -- ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Guido van Rossum
Guido van Rossum added the comment: OK let’s do it. Clearly for *some* applications the overhead is significant. -- --Guido (mobile) -- ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Ruslan Dautkhanov
Ruslan Dautkhanov added the comment: Thank you Guido and Ivan -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Ivan Levkivskyi
Ivan Levkivskyi added the comment: OK, here is the original issue https://github.com/python/typing/issues/681. I asked the author to open an issue here instead, but likely they didn't open one. -- ___ Python tracker

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Guido van Rossum
Guido van Rossum added the comment: If that solves the perf issue I am fine with it. -- ___ Python tracker ___ ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-31 Thread Ivan Levkivskyi
Ivan Levkivskyi added the comment: This issue came up few times before (although I can't find an issue here on b.p.o., maybe it was on typing-sig list). Although in micro-benchmarks the impact may seem big, in vast majority of applications it is rarely more that a percent or so. On the

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Guido van Rossum
Guido van Rossum added the comment: Hm, here's what I measure in Python 3.8.1. (I don't use IPython or notebooks so this looks a little different.) >>> timeit.timeit('Foo()', 'class Foo: pass') 0.3763025619934 >>> timeit.timeit('Foo()', 'class Foo:\n def __new__(cls): return >>>

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Guido van Rossum
Guido van Rossum added the comment: Sorry, you already said 3.6 and 3.8 give the same effect. But what if you add a minimal __new__() to Foo? -- ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Guido van Rossum
Guido van Rossum added the comment: What Python version was used for the timings? If not 3.8, please do over in 3.8. -- ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Ruslan Dautkhanov
Ruslan Dautkhanov added the comment: In [12]: cProfile.run("for _ in range(100_000): Bar()") 23 function calls in 0.136

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Ned Deily
Change by Ned Deily : -- nosy: +gvanrossum, levkivskyi ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Ruslan Dautkhanov
Ruslan Dautkhanov added the comment: Python typing gives an order of magnitude slow down in this case -- ___ Python tracker ___

[issue39168] Generic type subscription is a huge toll on Python performance

2019-12-30 Thread Ruslan Dautkhanov
l severity: normal status: open title: Generic type subscription is a huge toll on Python performance type: performance versions: Python 3.6, Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.o

Python performance

2014-03-08 Thread JCosta
I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Thanks -- https://mail.python.org/mailman/listinfo/python-list

Re: Python performance

2014-03-08 Thread Chris Angelico
On Sat, Mar 8, 2014 at 11:53 PM, JCosta generalco...@gmail.com wrote: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Thanks The first thing to look at is the conversion. If you convert

Re: Python performance

2014-03-08 Thread Tim Chase
On 2014-03-08 04:53, JCosta wrote: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? It depends. Did you write C#/Java in Python (i.e., use C# or Java idioms in Python), or did you write

Re: Python performance

2014-03-08 Thread Marko Rauhamaa
JCosta generalco...@gmail.com: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Yes. The main reason is the dot notation, which in C through Java is implemented by the compiler as a fixed

Re: Python performance

2014-03-08 Thread JCosta
Sábado, 8 de Março de 2014 12:53:57 UTC, JCosta escreveu: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Thanks ... Thanks for the help (Chris, Tim and Marko) and it´s

Re: Python performance

2014-03-08 Thread Mark Lawrence
On 08/03/2014 18:30, JCosta wrote: Sábado, 8 de Março de 2014 12:53:57 UTC, JCosta escreveu: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Thanks ... Thanks for the help

Re: Python performance

2014-03-08 Thread Ned Batchelder
On 3/8/14 7:53 AM, JCosta wrote: I did some work in c# and java and I converted some application to Python; I noticed Python is much slower than the other languages. Is this normal ? Thanks Your question, and the replies so far in this thread, have overlooked the difference between

Re: Python performance

2013-08-03 Thread Wayne Werner
On Fri, 2 Aug 2013, Schneider wrote: Hi list, I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000 mails per hour for at least

Re: Python performance

2013-08-03 Thread Stefan Behnel
Wayne Werner, 03.08.2013 15:09: On Fri, 2 Aug 2013, Schneider wrote: I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000 mails

Python performance

2013-08-02 Thread Schneider
Hi list, I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000 mails per hour for at least 8hours a day (which does not mean, that

Re: Python performance

2013-08-02 Thread Tim Chase
On 2013-08-02 14:00, Schneider wrote: I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000 mails per hour for at least 8hours a

Re: Python performance

2013-08-02 Thread Schneider
On Fri 02 Aug 2013 02:59:26 PM CEST, Tim Chase wrote: On 2013-08-02 14:00, Schneider wrote: I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle

Re: Python performance

2013-08-02 Thread Ray Cote
- Original Message - From: Schneider j...@globe.de To: python-list@python.org Sent: Friday, August 2, 2013 8:00:09 AM Subject: Python performance Hi list, I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python

Re: Python performance

2013-08-02 Thread Schneider
On 08/02/2013 03:09 PM, Ray Cote wrote: - Original Message - From: Schneider j...@globe.de To: python-list@python.org Sent: Friday, August 2, 2013 8:00:09 AM Subject: Python performance Hi list, I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering

Re: Python performance

2013-08-02 Thread Chris Angelico
On Fri, Aug 2, 2013 at 2:16 PM, Schneider j...@globe.de wrote: Queuing the mails for a while is not possible, because the tool should sit between the client and smtp-server. It should act as proxy, not as server. I've written an SMTP proxy (primary purpose: check SPF records; secondary

Re: Python performance

2013-08-02 Thread Grant Edwards
On 2013-08-02, Schneider j...@globe.de wrote: I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000 mails per hour for at least

Re: Python performance

2013-08-02 Thread Grant Edwards
On 2013-08-02, Grant Edwards invalid@invalid.invalid wrote: On 2013-08-02, Schneider j...@globe.de wrote: I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ).

Re: Python performance

2013-08-02 Thread Chris Angelico
On Fri, Aug 2, 2013 at 1:00 PM, Schneider j...@globe.de wrote: Hi list, I have to write a small SMTP-Relay script (+ some statistic infos) and I'm wondering, if this can be done in python (in terms of performance, of course not in terms of possibility ;) ). It has to handle around 2000

Re: improving python performance by extension module (64bit)

2010-06-26 Thread Nobody
On Fri, 25 Jun 2010 20:08:27 -0400, geremy condra wrote: I have written Haskell that runs faster than C, and Forth that runs faster than C, Faster than *what* C, though? With Haskell, there's seldom a significant performance hit for using -fvia-C, so you would probably have been able to get

Re: improving python performance by extension module (64bit)

2010-06-26 Thread geremy condra
On Sat, Jun 26, 2010 at 7:11 AM, Nobody nob...@nowhere.com wrote: On Fri, 25 Jun 2010 20:08:27 -0400, geremy condra wrote: I have written Haskell that runs faster than C, and Forth that runs faster than C, Faster than *what* C, though? Well, than the C it was replacing, which is admittedly

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Tim Wintle
On Thu, 2010-06-24 at 21:52 -0500, Peng Yu wrote: http://psyco.sourceforge.net/ The above package can improve python program on 32 bit library. But I need to run on 64 bit library. Is there any other module that can help improving the performance of python on 64 bit? As I understand it,

Re: improving python performance by extension module (64bit)

2010-06-25 Thread geremy condra
On Fri, Jun 25, 2010 at 7:01 AM, Tim Wintle tim.win...@teamrubber.com wrote: On Thu, 2010-06-24 at 21:52 -0500, Peng Yu wrote: http://psyco.sourceforge.net/ The above package can improve python program on 32 bit library. But I need to run on 64 bit library. Is there any other module that can

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Stephen Hansen
On Thu, Jun 24, 2010 at 7:52 PM, Peng Yu pengyu...@gmail.com wrote: http://psyco.sourceforge.net/ The above package can improve python program on 32 bit library. But I need to run on 64 bit library. Is there any other module that can help improving the performance of python on 64 bit? This

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Mark Lawrence
On 25/06/2010 16:34, Stephen Hansen wrote: Python's slow, sure. But its in practice fast enough for an extremely broad range of activities. What? Kindest regards. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Stephen Hansen
On Fri, Jun 25, 2010 at 1:51 PM, Mark Lawrence breamore...@yahoo.co.ukwrote: On 25/06/2010 16:34, Stephen Hansen wrote: Python's slow, sure. But its in practice fast enough for an extremely broad range of activities. What? What, what? --S --

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Mark Lawrence
On 25/06/2010 22:25, Stephen Hansen wrote: On Fri, Jun 25, 2010 at 1:51 PM, Mark Lawrencebreamore...@yahoo.co.ukwrote: On 25/06/2010 16:34, Stephen Hansen wrote: Python's slow, sure. But its in practice fast enough for an extremely broad range of activities. What? What, what? --S

Re: improving python performance by extension module (64bit)

2010-06-25 Thread Stephen Hansen
On Fri, Jun 25, 2010 at 3:18 PM, Mark Lawrence breamore...@yahoo.co.ukwrote: On 25/06/2010 22:25, Stephen Hansen wrote: On Fri, Jun 25, 2010 at 1:51 PM, Mark Lawrencebreamore...@yahoo.co.uk wrote: On 25/06/2010 16:34, Stephen Hansen wrote: Python's slow, sure. But its in practice fast

Re: improving python performance by extension module (64bit)

2010-06-25 Thread geremy condra
On Fri, Jun 25, 2010 at 6:18 PM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 25/06/2010 22:25, Stephen Hansen wrote: On Fri, Jun 25, 2010 at 1:51 PM, Mark Lawrencebreamore...@yahoo.co.ukwrote: On 25/06/2010 16:34, Stephen Hansen wrote:  Python's slow, sure. But its in practice fast

improving python performance by extension module (64bit)

2010-06-24 Thread Peng Yu
http://psyco.sourceforge.net/ The above package can improve python program on 32 bit library. But I need to run on 64 bit library. Is there any other module that can help improving the performance of python on 64 bit? -- Regards, Peng -- http://mail.python.org/mailman/listinfo/python-list

Re: Overcoming python performance penalty for multicore CPU

2010-02-21 Thread Martin v. Loewis
John Nagle wrote: I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if the GIL just limited Python to running on one CPU at a time, but it's worse than

Re: Overcoming python performance penalty for multicore CPU

2010-02-21 Thread Ryan Kelly
On Sun, 2010-02-21 at 22:22 +0100, Martin v. Loewis wrote: John Nagle wrote: I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if the GIL just limited

Re: Overcoming python performance penalty for multicore CPU

2010-02-21 Thread Martin v. Loewis
It's far from scientific, but I've seen behaviour that's close to a 100% performance penalty on a dual-core linux system: http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2 Short story: a particular test suite of mine used to run in around 25 seconds, but a bit of ctypes magic

Re: Overcoming python performance penalty for multicore CPU

2010-02-21 Thread Ryan Kelly
On Sun, 2010-02-21 at 23:05 +0100, Martin v. Loewis wrote: It's far from scientific, but I've seen behaviour that's close to a 100% performance penalty on a dual-core linux system: http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2 Short story: a particular test suite of

Re: Overcoming python performance penalty for multicore CPU

2010-02-08 Thread Stefan Behnel
Paul Rubin, 04.02.2010 02:51: John Nagle writes: Analysis of each domain is performed in a separate process, but each process uses multiple threads to read process several web pages simultaneously. Some of the threads go compute-bound for a second or two at a time as they parse web

Re: Overcoming python performance penalty for multicore CPU

2010-02-08 Thread Paul Rubin
Stefan Behnel stefan...@behnel.de writes: Well, if multi-core performance is so important here, then there's a pretty simple thing the OP can do: switch to lxml. http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ Well, lxml is uses libxml2, a fast XML parser written in C,

Re: Overcoming python performance penalty for multicore CPU

2010-02-08 Thread Antoine Pitrou
Le Tue, 02 Feb 2010 15:02:49 -0800, John Nagle a écrit : I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if the GIL just limited Python to running on one

Re: Overcoming python performance penalty for multicore CPU

2010-02-08 Thread J Kenneth King
Paul Rubin no.em...@nospam.invalid writes: Stefan Behnel stefan...@behnel.de writes: Well, if multi-core performance is so important here, then there's a pretty simple thing the OP can do: switch to lxml. http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ Well, lxml is

Re: Overcoming python performance penalty for multicore CPU

2010-02-08 Thread John Krukoff
On Mon, 2010-02-08 at 01:10 -0800, Paul Rubin wrote: Stefan Behnel stefan...@behnel.de writes: Well, if multi-core performance is so important here, then there's a pretty simple thing the OP can do: switch to lxml. http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/

Re: Overcoming python performance penalty for multicore CPU

2010-02-04 Thread Paul Rubin
John Nagle na...@animats.com writes: Analysis of each domain is performed in a separate process, but each process uses multiple threads to read process several web pages simultaneously. Some of the threads go compute-bound for a second or two at a time as they parse web pages. You're

Re: Overcoming python performance penalty for multicore CPU

2010-02-04 Thread Paul Rubin
John Nagle na...@animats.com writes: There's enough intercommunication between the threads working on a single site that it's a pain to do them as subprocesses. And I definitely don't want to launch subprocesses for each page; the Python load time would be worse than the actual work. The

Re: Overcoming python performance penalty for multicore CPU

2010-02-04 Thread Anh Hai Trinh
On Feb 4, 10:46 am, John Nagle na...@animats.com wrote:     There's enough intercommunication between the threads working on a single site that it's a pain to do them as subprocesses. And I definitely don't want to launch subprocesses for each page; the Python load time would be worse than

Re: Overcoming python performance penalty for multicore CPU

2010-02-03 Thread John Nagle
Paul Rubin wrote: John Nagle na...@animats.com writes: Analysis of each domain is performed in a separate process, but each process uses multiple threads to read process several web pages simultaneously. Some of the threads go compute-bound for a second or two at a time as they parse web

Re: Overcoming python performance penalty for multicore CPU

2010-02-03 Thread Steve Holden
John Nagle wrote: Paul Rubin wrote: John Nagle na...@animats.com writes: Analysis of each domain is performed in a separate process, but each process uses multiple threads to read process several web pages simultaneously. Some of the threads go compute-bound for a second or two at a time

Re: Overcoming python performance penalty for multicore CPU

2010-02-03 Thread John Nagle
Steve Holden wrote: John Nagle wrote: Paul Rubin wrote: John Nagle na...@animats.com writes: Analysis of each domain is performed in a separate process, but each process uses multiple threads to read process several web pages simultaneously. Some of the threads go compute-bound for a

Overcoming python performance penalty for multicore CPU

2010-02-02 Thread John Nagle
I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if the GIL just limited Python to running on one CPU at a time, but it's worse than that; there's excessive

Re: Overcoming python performance penalty for multicore CPU

2010-02-02 Thread exarkun
On 11:02 pm, na...@animats.com wrote: I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if the GIL just limited Python to running on one CPU at a time, but

Re: Overcoming python performance penalty for multicore CPU

2010-02-02 Thread alex23
On Feb 3, 9:02 am, John Nagle na...@animats.com wrote:     I know there's a performance penalty for running Python on a multicore CPU, but how bad is it?  I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It's a shame that Python 3.x is dead to you, otherwise you'd be able to

Re: Overcoming python performance penalty for multicore CPU

2010-02-02 Thread Terry Reedy
On 2/2/2010 9:02 PM, alex23 wrote: On Feb 3, 9:02 am, John Naglena...@animats.com wrote: I know there's a performance penalty for running Python on a multicore CPU, but how bad is it? I've read the key paper (www.dabeaz.com/python/GIL.pdf), of course. It's a shame that Python 3.x is

Re: python performance on Solaris

2009-10-16 Thread Dieter Maurer
Antoine Pitrou solip...@pitrou.net writes on Thu, 15 Oct 2009 16:25:43 + (UTC): Le Wed, 14 Oct 2009 22:39:14 -0700, John Nagle a écrit : Note that multithreaded compute-bound Python programs really suck on multiprocessors. Adding a second CPU makes the program go slower, due to

Re: python performance on Solaris

2009-10-15 Thread Antoine Pitrou
Le Wed, 14 Oct 2009 22:39:14 -0700, John Nagle a écrit : Note that multithreaded compute-bound Python programs really suck on multiprocessors. Adding a second CPU makes the program go slower, due to a lame mechanism for resolving conflicts over the global interpreter lock. I'm not sure

Re: python performance on Solaris

2009-10-15 Thread Aahz
In article 1a4707f5-85be-4f5f-ac3e-cf8f5bd21...@b15g2000yqd.googlegroups.com, inaf cem.ezbe...@gmail.com wrote: I have been following this group for quite some time and I figured (after searching enough on google --and on this group-- and not finding anything useful) I could pose this question

Re: python performance on Solaris

2009-10-15 Thread Adam Vande More
On Thu, Oct 15, 2009 at 4:35 PM, Aahz a...@pythoncraft.com wrote: In article 1a4707f5-85be-4f5f-ac3e-cf8f5bd21...@b15g2000yqd.googlegroups.com, inaf cem.ezbe...@gmail.com wrote: I have been following this group for quite some time and I figured (after searching enough on google --and on

Re: python performance on Solaris

2009-10-14 Thread Antoine Pitrou
inaf cem.ezberci at gmail.com writes: Good point. I failed to compare the CPU power on these machines.. 32 bit linux box I have is 2666 Mhz vs the Solaris zone is 1415 Mhz.. I guess that explains :) Thank you for the tip.. You have to compare not only CPU frequencies but the CPU models.

Re: python performance on Solaris

2009-10-14 Thread Jorgen Grahn
On Wed, 2009-10-14, Antoine Pitrou wrote: inaf cem.ezberci at gmail.com writes: Good point. I failed to compare the CPU power on these machines.. 32 bit linux box I have is 2666 Mhz vs the Solaris zone is 1415 Mhz.. I guess that explains :) Thank you for the tip.. You have to compare not

Re: python performance on Solaris

2009-10-14 Thread James Matthews
I use python in almost the same environment. I use it on Joyent and on the Rackspace cloud. Joyent is faster for a few reasons (cpu bursting and faster disks) but these aren't real benchmarks until they are on the same machines. James On Wed, Oct 14, 2009 at 9:59 AM, Jorgen Grahn

Re: python performance on Solaris

2009-10-14 Thread inaf
On Oct 14, 7:15 am, Antoine Pitrou solip...@pitrou.net wrote: inaf cem.ezberci at gmail.com writes: Good point. I failed to compare the CPU power on these machines.. 32 bit linux box I have is 2666 Mhz vs the Solaris zone is 1415 Mhz.. I guess that explains :) Thank you for the tip..

Re: python performance on Solaris

2009-10-14 Thread John Nagle
inaf wrote: I have been following this group for quite some time and I figured (after searching enough on google --and on this group-- and not finding anything useful) I could pose this question here. Can anyone shed some light on python's performance on Solaris? Note that multithreaded

Re: python performance on Solaris

2009-10-11 Thread Antoine Pitrou
inaf cem.ezberci at gmail.com writes: My code seem to return lookups from a in memory data structure I build combining bunch of dictionaries and lists 6-8 times faster on a 32 bit Linux box than on a Solaris zone. Well, if your workload is CPU-bound, the issue here is not really Solaris vs.

Re: python performance on Solaris

2009-10-11 Thread inaf
On Oct 11, 6:59 am, Antoine Pitrou solip...@pitrou.net wrote: inaf cem.ezberci at gmail.com writes: My code seem to return lookups from a in memory data structure I build combining bunch of dictionaries and lists 6-8 times faster on a 32 bit Linux box than on a Solaris zone. Well, if

Re: A question on python performance.

2007-09-28 Thread Magnus Lycka
Joe Goldthwaite wrote: I didn't know about the getattr function. I tried to search for that type of function but not knowing how to word the search request, I couldn't find it. You should really read through chapter 2 (Built-in Objects) of the library reference. All that stuff is core Python

Re: A question on python performance.

2007-09-28 Thread Bruno Desthuilliers
Joe Goldthwaite a écrit : (snip) I guess I still think of decorators as the people who got the gym ready for the prom. I've tried getting up to speed on decorators but I haven't had much success. The @decorator syntax is just syntactic sugar for a common use of higher order functions. So

RE: A question on python performance.

2007-09-27 Thread Joe Goldthwaite
[EMAIL PROTECTED] wrote: Makes perfect sense to me! Think about it: method 1: looks up the method directly from the object (fastest) method 2: looks up __class__, then looks up __dict__, then gets the element from __dict__ method 3: looks up caller, looks up __class__, looks up __dict__, gets

A question on python performance.

2007-09-26 Thread Joe Goldthwaite
Hi everyone, I'm a developer who's been using python for a couple of years. I wrote a fairly large application using it but I was learning the language at the same time so it most of the code kind of sucks. I've learned a lot since then and I've been going through my code trying to organize it

Re: A question on python performance.

2007-09-26 Thread chris . monsanto
On Sep 26, 2:26 pm, Joe Goldthwaite [EMAIL PROTECTED] wrote: Hi everyone, I'm a developer who's been using python for a couple of years. I wrote a fairly large application using it but I was learning the language at the same time so it most of the code kind of sucks. I've learned a lot

Re: A question on python performance.

2007-09-26 Thread Erik Jones
On Sep 26, 2007, at 1:26 PM, Joe Goldthwaite wrote: Hi everyone, I'm a developer who's been using python for a couple of years. I wrote a fairly large application using it but I was learning the language at the same time so it most of the code kind of sucks. I've learned a lot since

Re: A question on python performance.

2007-09-26 Thread Paul Hankin
On Sep 26, 7:26 pm, Joe Goldthwaite [EMAIL PROTECTED] wrote: The code gets kind of wordy so I started trying to figure out how to call them dynamically since the param type is the same as the method the retrieves it. I came up with this; def getValue(trend, param, per): return

Re: A question on python performance.

2007-09-26 Thread Bruno Desthuilliers
Joe Goldthwaite a écrit : Hi everyone, I'm a developer who's been using python for a couple of years. I wrote a fairly large application using it but I was learning the language at the same time so it most of the code kind of sucks. I've learned a lot since then and I've been going

Python performance tips page moved to wiki

2005-03-29 Thread Skip Montanaro
I dumped my old fastpython.html web page: http://manatee.mojam.com/~skip/python/fastpython.html in favor of a page on the Python wiki: http://www.python.org/moin/PythonSpeed/PerformanceTips Now everybody can help fix warts, add content, etc, etc, etc. References to the old page are

  1   2   >