Re: [pypy-dev] Scrapy fails in PyPy

2012-12-12 Thread Alex Gaynor
Out of curiosity Stefan, if we had an alternate C-API with similar methods (e.g. PyPyList_Append or so), but different signatures and memory model, how hard do you think it would be to have Cython support this? Alex On Wed, Dec 12, 2012 at 11:35 PM, Stefan Behnel wrote: > Maciej Fijalkowski, 1

Re: [pypy-dev] Scrapy fails in PyPy

2012-12-12 Thread Stefan Behnel
Maciej Fijalkowski, 12.12.2012 20:10: > On Wed, Dec 12, 2012 at 7:06 PM, Joe Hillenbrand wrote: >> I was able to fix the issue with scrapy. >> >> https://github.com/joehillen/scrapy/commit/8778af5c5be50a5d746751352f8d710d1f24681c >> >> Unfortunately, scrapy takes twice as long in PyPy than in CPyth

Re: [pypy-dev] CFFI speed results

2012-12-12 Thread Elefterios Stamatogiannakis
We cannot publish the benchmark that we used due to its usage of non public data. We'll prepare a benchmark which uses synthetic data, nevertheless it'll have similar performance characteristics to the previous one, and we'll provide it to you. Many many thanks, l. On 11/12/2012 7:48 μμ, M

Re: [pypy-dev] Scrapy fails in PyPy

2012-12-12 Thread Maciej Fijalkowski
On Wed, Dec 12, 2012 at 7:06 PM, Joe Hillenbrand wrote: > I was able to fix the issue with scrapy. > > https://github.com/joehillen/scrapy/commit/8778af5c5be50a5d746751352f8d710d1f24681c > > Unfortunately, scrapy takes twice as long in PyPy than in CPython. I suspect > this is because lxml is twic

Re: [pypy-dev] Scrapy fails in PyPy

2012-12-12 Thread Joe Hillenbrand
I was able to fix the issue with scrapy. https://github.com/joehillen/scrapy/commit/8778af5c5be50a5d746751352f8d710d1f24681c Unfortunately, scrapy takes twice as long in PyPy than in CPython. I suspect this is because lxml is twice as slow in PyPy vs CPython, which I found in lxml's benchmarks.

Re: [pypy-dev] Raymond's proposed new dict layout

2012-12-12 Thread Carl Friedrich Bolz
On 12/12/2012 03:11 PM, Armin Rigo wrote: Hi Carl Friedrich, On Wed, Dec 12, 2012 at 3:00 PM, Carl Friedrich Bolz wrote: Ah, cool. Guess I should join the channel again :-). When was that? Couldn't find the logs. The 10th, 17:31 (http://www.tismer.com/pypy/irc-logs/pypy/pypy.2012-12-10.log).

Re: [pypy-dev] Raymond's proposed new dict layout

2012-12-12 Thread Armin Rigo
Hi Carl Friedrich, On Wed, Dec 12, 2012 at 3:00 PM, Carl Friedrich Bolz wrote: > Ah, cool. Guess I should join the channel again :-). When was that? > Couldn't find the logs. The 10th, 17:31 (http://www.tismer.com/pypy/irc-logs/pypy/pypy.2012-12-10.log). A bientôt, Armin.

Re: [pypy-dev] Raymond's proposed new dict layout

2012-12-12 Thread Carl Friedrich Bolz
On 12/12/2012 02:26 PM, Armin Rigo wrote: Hi Carl Friedrich, On Wed, Dec 12, 2012 at 2:03 PM, Carl Friedrich Bolz wrote: I've been wondering whether the recipe that Raymond Hettinger posted would also solve PyPy's performance problems with making huge dictionaries: Yes, we've been wondering

Re: [pypy-dev] Raymond's proposed new dict layout

2012-12-12 Thread Armin Rigo
Hi Carl Friedrich, On Wed, Dec 12, 2012 at 2:03 PM, Carl Friedrich Bolz wrote: > I've been wondering whether the recipe that Raymond Hettinger posted > would also solve PyPy's performance problems with making huge > dictionaries: Yes, we've been wondering the same thing on IRC :-) Armin ___

[pypy-dev] Raymond's proposed new dict layout

2012-12-12 Thread Carl Friedrich Bolz
Hi all, I've been wondering whether the recipe that Raymond Hettinger posted would also solve PyPy's performance problems with making huge dictionaries: http://code.activestate.com/recipes/578375/ The proposed layout makes the pointer-storing part of the dictionary behave much more like a list.