On Sun, 2010-02-21 at 23:05 +0100, Martin v. Loewis wrote:
> > It's far from scientific, but I've seen behaviour that's close to a 100%
> > performance penalty on a dual-core linux system:
> >
> >http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2
> >
> > Short story: a particular te
> It's far from scientific, but I've seen behaviour that's close to a 100%
> performance penalty on a dual-core linux system:
>
>http://www.rfk.id.au/blog/entry/a-gil-adventure-threading2
>
> Short story: a particular test suite of mine used to run in around 25
> seconds, but a bit of ctypes
On Sun, 2010-02-21 at 22:22 +0100, Martin v. Loewis wrote:
> John Nagle wrote:
> >I know there's a performance penalty for running Python on a
> > multicore CPU, but how bad is it? I've read the key paper
> > ("www.dabeaz.com/python/GIL.pdf"), of course. It would be adequate
> > if the GIL ju
John Nagle wrote:
>I know there's a performance penalty for running Python on a
> multicore CPU, but how bad is it? I've read the key paper
> ("www.dabeaz.com/python/GIL.pdf"), of course. It would be adequate
> if the GIL just limited Python to running on one CPU at a time,
> but it's worse t
On Mon, 2010-02-08 at 01:10 -0800, Paul Rubin wrote:
> Stefan Behnel writes:
> > Well, if multi-core performance is so important here, then there's a pretty
> > simple thing the OP can do: switch to lxml.
> >
> > http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
>
> Well, lxml
Paul Rubin writes:
> Stefan Behnel writes:
>> Well, if multi-core performance is so important here, then there's a pretty
>> simple thing the OP can do: switch to lxml.
>>
>> http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
>
> Well, lxml is uses libxml2, a fast XML parser w
Le Tue, 02 Feb 2010 15:02:49 -0800, John Nagle a écrit :
> I know there's a performance penalty for running Python on a multicore
> CPU, but how bad is it? I've read the key paper
> ("www.dabeaz.com/python/GIL.pdf"), of course. It would be adequate if
> the GIL just limited Python to running on o
Stefan Behnel writes:
> Well, if multi-core performance is so important here, then there's a pretty
> simple thing the OP can do: switch to lxml.
>
> http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
Well, lxml is uses libxml2, a fast XML parser written in C, but AFAIK it
only
Paul Rubin, 04.02.2010 02:51:
> John Nagle writes:
>> Analysis of each domain is
>> performed in a separate process, but each process uses multiple
>> threads to read process several web pages simultaneously.
>>
>>Some of the threads go compute-bound for a second or two at a time as
>> they par
On Feb 4, 10:46 am, John Nagle wrote:
>
> There's enough intercommunication between the threads working on
> a single site that it's a pain to do them as subprocesses. And I
> definitely don't want to launch subprocesses for each page; the
> Python load time would be worse than the actual work
John Nagle writes:
>There's enough intercommunication between the threads working on
> a single site that it's a pain to do them as subprocesses. And I
> definitely don't want to launch subprocesses for each page; the
> Python load time would be worse than the actual work. The
> subprocess mo
John Nagle writes:
> Analysis of each domain is
> performed in a separate process, but each process uses multiple
> threads to read process several web pages simultaneously.
>
>Some of the threads go compute-bound for a second or two at a time as
> they parse web pages.
You're probably bett
Steve Holden wrote:
John Nagle wrote:
Paul Rubin wrote:
John Nagle writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a tim
John Nagle wrote:
> Paul Rubin wrote:
>> John Nagle writes:
>>> Analysis of each domain is
>>> performed in a separate process, but each process uses multiple
>>> threads to read process several web pages simultaneously.
>>>
>>>Some of the threads go compute-bound for a second or two at a time
Paul Rubin wrote:
John Nagle writes:
Analysis of each domain is
performed in a separate process, but each process uses multiple
threads to read process several web pages simultaneously.
Some of the threads go compute-bound for a second or two at a time as
they parse web pages.
You're pr
On 2/2/2010 9:02 PM, alex23 wrote:
On Feb 3, 9:02 am, John Nagle wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
("www.dabeaz.com/python/GIL.pdf"), of course.
It's a shame that Python 3.x is dead to you, other
On Feb 3, 9:02 am, John Nagle wrote:
> I know there's a performance penalty for running Python on a
> multicore CPU, but how bad is it? I've read the key paper
> ("www.dabeaz.com/python/GIL.pdf"), of course.
It's a shame that Python 3.x is dead to you, otherwise you'd be able
to enjoy the ne
On 11:02 pm, na...@animats.com wrote:
I know there's a performance penalty for running Python on a
multicore CPU, but how bad is it? I've read the key paper
("www.dabeaz.com/python/GIL.pdf"), of course. It would be adequate
if the GIL just limited Python to running on one CPU at a time,
but
18 matches
Mail list logo
