John Nagle wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course. It would be adequate
if the GIL just limited Python to running on one CPU at a time,
but it's worse than
On Sun, 2010-02-21 at 22:22 +0100, Martin v. Loewis wrote:
John Nagle wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course. It would be adequate
if the GIL just limited
It's far from scientific, but I've seen behaviour that's close to a 100%
performance penalty on a dual-core linux system:
http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2
Short story: a particular test suite of mine used to run in around 25
seconds, but a bit of ctypes magic
On Sun, 2010-02-21 at 23:05 +0100, Martin v. Loewis wrote:
It's far from scientific, but I've seen behaviour that's close to a 100%
performance penalty on a dual-core linux system:
http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2
Short story: a particular test suite of
Paul Rubin, 04.02.2010 02:51:
John Nagle writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a time as
they parse web
Stefan Behnel stefan...@behnel.de writes:
Well, if multi-core performance is so important here, then there's a pretty
simple thing the OP can do: switch to lxml.
http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
Well, lxml is uses libxml2, a fast XML parser written in C,
Le Tue, 02 Feb 2010 15:02:49 -0800, John Nagle a écrit :
I know there's a performance penalty for running Python on a multicore
CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course. It would be adequate if
the GIL just limited Python to running on one
Paul Rubin no.em...@nospam.invalid writes:
Stefan Behnel stefan...@behnel.de writes:
Well, if multi-core performance is so important here, then there's a pretty
simple thing the OP can do: switch to lxml.
http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
Well, lxml is
On Mon, 2010-02-08 at 01:10 -0800, Paul Rubin wrote:
Stefan Behnel stefan...@behnel.de writes:
Well, if multi-core performance is so important here, then there's a pretty
simple thing the OP can do: switch to lxml.
http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
John Nagle na...@animats.com writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a time as
they parse web pages.
You're
John Nagle na...@animats.com writes:
There's enough intercommunication between the threads working on
a single site that it's a pain to do them as subprocesses. And I
definitely don't want to launch subprocesses for each page; the
Python load time would be worse than the actual work. The
On Feb 4, 10:46 am, John Nagle na...@animats.com wrote:
There's enough intercommunication between the threads working on
a single site that it's a pain to do them as subprocesses. And I
definitely don't want to launch subprocesses for each page; the
Python load time would be worse than
Paul Rubin wrote:
John Nagle na...@animats.com writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a time as
they parse web
John Nagle wrote:
Paul Rubin wrote:
John Nagle na...@animats.com writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a time
Steve Holden wrote:
John Nagle wrote:
Paul Rubin wrote:
John Nagle na...@animats.com writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course. It would be adequate
if the GIL just limited Python to running on one CPU at a time,
but it's worse than that; there's excessive
On 11:02 pm, na...@animats.com wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course. It would be adequate
if the GIL just limited Python to running on one CPU at a time,
but
On Feb 3, 9:02 am, John Nagle na...@animats.com wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course.
It's a shame that Python 3.x is dead to you, otherwise you'd be able
to
On 2/2/2010 9:02 PM, alex23 wrote:
On Feb 3, 9:02 am, John Naglena...@animats.com wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
(www.dabeaz.com/python/GIL.pdf), of course.
It's a shame that Python 3.x is
19 matches
Mail list logo
