Re: [pypy-dev] CFFI speed results

2012-12-29 Thread Armin Rigo
Hi, On Thu, Dec 27, 2012 at 2:33 PM, Elefterios Stamatogiannakis wrote: > Also linux's 32 bit wchars are unfortunate in our case, because SQLite only > supports UTF-8 and UTF-16 in their API. So there is no way on linux to > directly pass pypy's unicode strings to SQLite. Yes, given that pypy st

Re: [pypy-dev] CFFI speed results

2012-12-27 Thread Leonardo Santagada
I might be wrong but cpython uses UCS internally on unicode, so even if sqlite had UTF-32 it would not work. On Thu, Dec 27, 2012 at 11:33 AM, Elefterios Stamatogiannakis < est...@gmail.com> wrote: > On 27/12/12 11:44, Armin Rigo wrote: > >> Normally, you should be able to pass a unicode object

Re: [pypy-dev] CFFI speed results

2012-12-27 Thread Elefterios Stamatogiannakis
On 27/12/12 11:44, Armin Rigo wrote: Normally, you should be able to pass a unicode object directly to a "wchar_t *" argument and get a complete wchar_t-encoded string. (Note that wchar_t is 16-bit on Windows but 32-bit on Linux, for example.) Armin thanks for the clarification. From your not

Re: [pypy-dev] CFFI speed results

2012-12-27 Thread Armin Rigo
Hi Elefterios, On Wed, Dec 26, 2012 at 4:24 PM, Elefterios Stamatogiannakis wrote: > The problem was that giving directly a pypy string to above function (using > wchar_t type), only the first character of the string was passed to SQLite. Do you mean you unexpectedly got only the first character

Re: [pypy-dev] CFFI speed results

2012-12-26 Thread Elefterios Stamatogiannakis
On 26/12/2012 6:19 μμ, Maciej Fijalkowski wrote: On Wed, Dec 26, 2012 at 5:24 PM, Elefterios Stamatogiannakis wrote: Thank you for looking into this part of pypy's performance. Whenever something reaches a testable state we would be glad to test/benchmark it. On another front. We tried using

Re: [pypy-dev] CFFI speed results

2012-12-26 Thread Maciej Fijalkowski
On Wed, Dec 26, 2012 at 5:24 PM, Elefterios Stamatogiannakis wrote: > On 26/12/2012 12:48 μμ, Maciej Fijalkowski wrote: >> >> On Mon, Dec 17, 2012 at 10:42 PM, Maciej Fijalkowski >> wrote: >>> >>> On Mon, Dec 17, 2012 at 3:46 PM, Eleytherios Stamatogiannakis >>> wrote: We had a "bug" i

Re: [pypy-dev] CFFI speed results

2012-12-26 Thread Elefterios Stamatogiannakis
On 26/12/2012 12:48 μμ, Maciej Fijalkowski wrote: On Mon, Dec 17, 2012 at 10:42 PM, Maciej Fijalkowski wrote: On Mon, Dec 17, 2012 at 3:46 PM, Eleytherios Stamatogiannakis wrote: We had a "bug" in our previous benchmark ("mspw_bench.sql"). The way it was written allowed SQLite to short-circui

Re: [pypy-dev] CFFI speed results

2012-12-26 Thread Maciej Fijalkowski
On Mon, Dec 17, 2012 at 10:42 PM, Maciej Fijalkowski wrote: > On Mon, Dec 17, 2012 at 3:46 PM, Eleytherios Stamatogiannakis > wrote: >> We had a "bug" in our previous benchmark ("mspw_bench.sql"). The way it was >> written allowed SQLite to short-circuit column data retrieval, ending up >> with m

Re: [pypy-dev] CFFI speed results

2012-12-17 Thread Maciej Fijalkowski
On Mon, Dec 17, 2012 at 3:46 PM, Eleytherios Stamatogiannakis wrote: > We had a "bug" in our previous benchmark ("mspw_bench.sql"). The way it was > written allowed SQLite to short-circuit column data retrieval, ending up > with minimal exercising of the CFFI layer. I thought it was a feature :)

Re: [pypy-dev] CFFI speed results

2012-12-17 Thread Eleytherios Stamatogiannakis
We had a "bug" in our previous benchmark ("mspw_bench.sql"). The way it was written allowed SQLite to short-circuit column data retrieval, ending up with minimal exercising of the CFFI layer. The attached query exercises CFFI as it should. We also checked its profiling characteristics, and it

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Elefterios Stamatogiannakis
On 15/12/2012 8:51 πμ, Maciej Fijalkowski wrote: On Sat, Dec 15, 2012 at 2:56 AM, Elefterios Stamatogiannakis wrote: On 15/12/2012 12:00 πμ, Maciej Fijalkowski wrote: Hi. For what is worth roughly 1/3 of the time is spent importing all the things. This is done in the compilation step in the

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Armin Rigo
Hi Bengt, On Sat, Dec 15, 2012 at 12:14 PM, Bengt Richter wrote: > I wonder whether something could be gained by having an alternative > internal unicode representation in the form of latin1 8-bit byte strings. This has been already implemented in CPython 3.x, based on earlier experiments on PyP

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Bengt Richter
On 12/15/2012 11:27 AM Armin Rigo wrote: Hi, On Sat, Dec 15, 2012 at 7:51 AM, Maciej Fijalkowski wrote: And ASPW does the same right? I understand the general need for UTF8, I just didn't find it in this particular query. Fwiw, I wonder again if we couldn't have all our unicode strings inter

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Armin Rigo
Hi, On Sat, Dec 15, 2012 at 12:00 PM, Maciej Fijalkowski wrote: > Some sort of string strategies? like "we know it's ascii" or so as well? Something like that, but that "strategy" would be the only one needed. We don't need ascii-only nor two-bytes-only strategies (nor, of course, 4-bytes strat

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Maciej Fijalkowski
On Sat, Dec 15, 2012 at 12:27 PM, Armin Rigo wrote: > Hi, > > On Sat, Dec 15, 2012 at 7:51 AM, Maciej Fijalkowski wrote: >> And ASPW does the same right? I understand the general need for UTF8, >> I just didn't find it in this particular query. > > Fwiw, I wonder again if we couldn't have all our

Re: [pypy-dev] CFFI speed results

2012-12-15 Thread Armin Rigo
Hi, On Sat, Dec 15, 2012 at 7:51 AM, Maciej Fijalkowski wrote: > And ASPW does the same right? I understand the general need for UTF8, > I just didn't find it in this particular query. Fwiw, I wonder again if we couldn't have all our unicode strings internally be UTF8 instead of 2- or 4-bytes st

Re: [pypy-dev] CFFI speed results

2012-12-14 Thread Maciej Fijalkowski
On Sat, Dec 15, 2012 at 2:56 AM, Elefterios Stamatogiannakis wrote: > On 15/12/2012 12:00 πμ, Maciej Fijalkowski wrote: >> >> Hi. >> >> For what is worth roughly 1/3 of the time is spent importing all the >> things. This is done in the compilation step in the ASPW, so please >> try running the sel

Re: [pypy-dev] CFFI speed results

2012-12-14 Thread Elefterios Stamatogiannakis
On 15/12/2012 12:00 πμ, Maciej Fijalkowski wrote: Hi. For what is worth roughly 1/3 of the time is spent importing all the things. This is done in the compilation step in the ASPW, so please try running the select few times. Another slightly worrying thing is that a lot of time is spent doing ut

Re: [pypy-dev] CFFI speed results

2012-12-14 Thread Elefterios Stamatogiannakis
On 15/12/2012 12:00 πμ, Maciej Fijalkowski wrote: Hi. For what is worth roughly 1/3 of the time is spent importing all the things. This is done in the compilation step in the ASPW, so please try running the select few times. Another slightly worrying thing is that a lot of time is spent doing ut

Re: [pypy-dev] CFFI speed results

2012-12-14 Thread Maciej Fijalkowski
On Fri, Dec 14, 2012 at 7:25 PM, Eleytherios Stamatogiannakis wrote: > Here is the synthetic benchmark. > > To run it you'll need latest madIS. You can clone it using: > > hg clone https://code.google.com/p/madis/ > > For running the test you can use: > > CPython 2.7 + APSW: > > https://code.googl

Re: [pypy-dev] CFFI speed results

2012-12-13 Thread Armin Rigo
Hi, On Tue, Dec 11, 2012 at 6:00 PM, Eleytherios Stamatogiannakis wrote: > Python/APSW: 40s > pypy/MSPW: 2m 3s > pypy/APSW: 2m 21s Not horribly bad, given that we're comparing with APSW, which is a piece of C code: PyPy makes Python only 3 times slower than hand-crafted C. That was just a ge

Re: [pypy-dev] CFFI speed results

2012-12-13 Thread Maciej Fijalkowski
On Wed, Dec 12, 2012 at 11:00 PM, Elefterios Stamatogiannakis wrote: > We cannot publish the benchmark that we used due to its usage of non public > data. > > We'll prepare a benchmark which uses synthetic data, nevertheless it'll have > similar performance characteristics to the previous one, and

Re: [pypy-dev] CFFI speed results

2012-12-12 Thread Elefterios Stamatogiannakis
We cannot publish the benchmark that we used due to its usage of non public data. We'll prepare a benchmark which uses synthetic data, nevertheless it'll have similar performance characteristics to the previous one, and we'll provide it to you. Many many thanks, l. On 11/12/2012 7:48 μμ, M

Re: [pypy-dev] CFFI speed results

2012-12-11 Thread Maciej Fijalkowski
On Tue, Dec 11, 2012 at 7:00 PM, Eleytherios Stamatogiannakis wrote: > Hello, > > We have been testing CFFI here for the purpose of speeding up madIS [*], and > here are some preliminary results. > > First of all, under pypy, CFFI is a *lot* faster than ctypes. In callback > microbenchmarks (using

[pypy-dev] CFFI speed results

2012-12-11 Thread Eleytherios Stamatogiannakis
Hello, We have been testing CFFI here for the purpose of speeding up madIS [*], and here are some preliminary results. First of all, under pypy, CFFI is a *lot* faster than ctypes. In callback microbenchmarks (using quicksort to call the callbacks), pypy/CFFI had ~8-10 times less overhead th