Hello,
We have been testing CFFI here for the purpose of speeding up madIS [*],
and here are some preliminary results.
First of all, under pypy, CFFI is a *lot* faster than ctypes. In
callback microbenchmarks (using quicksort to call the callbacks),
pypy/CFFI had ~8-10 times less overhead than pypy/ctypes.
Using libsqlite3 as the C library calling the callbacks, we found that,
compared to Python/APSW [**] the callback overhead of pypy/CFFI was 3-4
times larger.
To be able to run madIS, we started from pypy's _sqlite3.py (which uses
ctypes) and did a rough conversion of it to CFFI. We then changed it to
be APSW compatible. In the rest of the email, we'll refer to this APSW
compatible CFFI wrapper, as MSPW.
The end results were very encouraging. Using madIS and for SQL queries
that didn't produce a lot of columns, we could get speedups up to 2 times.
For SQL queries that produce a lot of columns, we get slowdowns of 2 -
2.5 times. Running a profiler on pypy/madIS, we found out that the main
bottleneck is not callback performance any more, but it is regular
pypy->C call speed.
Whereas Python/APSW spends most of its time on madIS' Python execution
of functions. Pypy/MSPW spends most of its time (~45-50%) on calling a
libsqlite3 function that passes a Python string back to libsqlite3, and
10% in pypy's string.encode('utf-8') function.
So for pypy/MSPW most of the time (55-60%) is spend just to pass a
string back to libsqlite3.
In Python/APSW the time spended to pass the string back to libsqlite3 is
<1%.
The libsqlite3 function's header info is:
void sqlite3_result_text(sqlite3_context*, const char*, int,
void(*)(void*));
In the main query that we've used in our tests, above libsqlite3
function is called 1.1 Million times.
The times of this query, running under the different options are:
Python/APSW: 40s
pypy/MSPW: 2m 3s
pypy/APSW: 2m 21s
Best regards,
l.
[*] https://code.google.com/p/madis/
[*] https://code.google.com/p/apsw/
_______________________________________________
pypy-dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-dev
